Yongjun Zhang
f7a27cdee4
HDFS-14118. Support using DNS to resolve nameservices to IP addresses. Contributed by Fengnan Li.
2019-02-23 09:35:36 -08:00
Daryn Sharp
a87e458432
HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-20 18:17:04 -08:00
Shweta Yakkali
371a6db59a
HDFS-14273. Fix checkstyle issues in BlockLocation's method javadoc
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I546aa4a0fe7f83b53735acd9925f366b2f1a00e2
2019-02-20 15:36:53 -08:00
Akira Ajisaka
dabfeab785
YARN-9308. fairscheduler-statedump.log gets generated regardless of service again after the merge of HDFS-7240. Contributed by Wilfred Spiegelenburg.
2019-02-15 14:49:49 +09:00
Erik Krogen
64f28f9efa
HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen.
2019-02-14 11:22:04 -08:00
Siyao Meng
fd026863b1
HDFS-14241. Provide feedback on successful renameSnapshot and deleteSnapshot. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-02-13 19:48:02 -08:00
Chen Liang
024c87291c
HDFS-13617. Allow wrapping NN QOP into token in encrypted message. Contributed by Chen Liang
2019-02-13 12:40:31 -08:00
Vinayakumar B
00c5ffaee2
HADOOP-16108. Tail Follow Interval Should Allow To Specify The Sleep Interval To Save Unnecessary RPC's. Contributed by Ayush Saxena.
2019-02-13 16:44:32 +05:30
Yiqun Lin
7b11b404a3
HADOOP-16097. Provide proper documentation for FairCallQueue. Contributed by Erik Krogen.
2019-02-13 11:16:04 +08:00
Steve Loughran
668817a6ce
Revert "HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth."
...
(accidentally mixed in two patches)
This reverts commit fa8cd1bf28
.
2019-02-07 21:57:22 +00:00
Wangda Tan
308f3168fa
Make upstream aware of 3.1.2 release
...
Change-Id: I397bc6ef75498726df4763bd07a8bf8fe1c38365
2019-02-05 14:03:18 -08:00
Da Zhou
fa8cd1bf28
HADOOP-15954. ABFS: Enable owner and group conversion for MSI and login user using OAuth.
...
Contributed by Da Zhou and Junhua Gu.
2019-02-05 19:23:15 +00:00
Steve Loughran
f365957c63
HADOOP-15229. Add FileSystem builder-based openFile() API to match createFile();
...
S3A to implement S3 Select through this API.
The new openFile() API is asynchronous, and implemented across FileSystem and FileContext.
The MapReduce V2 inputs are moved to this API, and you can actually set must/may
options to pass in.
This is more useful for setting things like s3a seek policy than for S3 select,
as the existing input format/record readers can't handle S3 select output where
the stream is shorter than the file length, and splitting plain text is suboptimal.
Future work is needed there.
In the meantime, any/all filesystem connectors are now free to add their own filesystem-specific
configuration parameters which can be set in jobs and used to set filesystem input stream
options (seek policy, retry, encryption secrets, etc).
Contributed by Steve Loughran
2019-02-05 11:51:02 +00:00
Steve Loughran
7f46d13dac
HADOOP-16079. Token.toString faulting if any token listed can't load.
...
Contributed by Steve Loughran.
2019-02-01 14:31:47 +00:00
Inigo Goiri
bcc3a79f58
HADOOP-16084. Fix the comment for getClass in Configuration. Contributed by Fengnan Li.
2019-01-31 10:06:05 -08:00
Akira Ajisaka
1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki.
2019-01-29 18:29:56 -08:00
Hanisha Koneru
b3bc94ebfd
HDFS-14236. Lazy persist copy/ put fails with ViewFs.
2019-01-29 16:45:44 -08:00
Wei-Chiu Chuang
d1714c20e9
Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
...
This reverts commit 1d523279da
.
2019-01-29 15:43:09 -08:00
Pranay Singh
1d523279da
HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-25 09:02:57 -08:00
Eric Yang
0dd35e218f
HADOOP-15922. Fixed DelegationTokenAuthenticator URL decoding for doAs user.
...
Contributed by He Xiaoqiao
2019-01-22 18:59:36 -05:00
Akira Ajisaka
a463cf75a0
HADOOP-15787. [JDK11] TestIPC.testRTEDuringConnectionSetup fails. Contributed by Zsolt Venczel.
2019-01-22 10:19:05 +09:00
Sunil G
2e2508b8e3
Make 3.2.0 aware to other branches
2019-01-21 21:24:51 +05:30
Surendra Singh Lilhore
dacc1a759e
HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena.
2019-01-18 13:55:12 +05:30
Akira Ajisaka
01405df87f
HADOOP-15941. Addendum patch. Contributed by Takanobu Asanuma.
2019-01-16 10:28:08 +09:00
luhuachao
ccc43621ab
HADOOP-16019. ZKDelegationTokenSecretManager won't log exception message occured in function setJaasConfiguration.
...
Contributed by luhuachao.
2019-01-15 13:15:18 +00:00
Steve Loughran
6d0bffe17e
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran and Daryn Sharp.
2019-01-14 17:59:27 +00:00
Inigo Goiri
35fa3bd685
HADOOP-16045. Don't run TestDU on Windows. Contributed by Lukas Majercak.
2019-01-11 18:07:19 -08:00
Chen Liang
bf08f4abae
HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian.
2019-01-11 14:01:23 -08:00
Chen Liang
01cb958af4
HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen.
2019-01-11 12:51:07 -08:00
Giovanni Matteo Fumarola
fb8932a727
HADOOP-16029. Consecutive StringBuilder.append can be reused. Contributed by Ayush Saxena.
2019-01-11 10:54:49 -08:00
Akira Ajisaka
d4ca907da6
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper
2019-01-11 11:24:05 +09:00
Kihwal Lee
33c009a45d
HADOOP-15909. KeyProvider class should implement Closeable. Contributed
...
by Kuhu Shukla.
2019-01-10 13:21:37 -06:00
Jason Lowe
c634589ab2
Revert "HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh."
...
This reverts commit ecdeaa7e6a
.
2019-01-09 17:24:58 -06:00
Takanobu Asanuma
709ddb10e5
HADOOP-15941. [JDK 11] Compilation failure: package com.sun.jndi.ldap is not visible.
2019-01-09 13:55:25 +09:00
shwetayakkali@cloudera.com
4ab5260b7e
HDFS-14132. Add BlockLocation.isStriped() to determine if block is replicated or Striped
...
(Contributed by Shweta Yakkali via Daniel Templeton)
Change-Id: I0ed8996a0bae2ad2c7d3513143195533f7191af8
2019-01-08 17:04:57 -08:00
Akira Ajisaka
7f78397036
Revert "HADOOP-14556. S3A to support Delegation Tokens."
...
This reverts commit d7152332b3
.
2019-01-08 14:51:30 +09:00
Steve Loughran
d7152332b3
HADOOP-14556. S3A to support Delegation Tokens.
...
Contributed by Steve Loughran.
2019-01-07 13:18:03 +00:00
Eric Yang
d43af8b3db
HADOOP-15996. Improved Kerberos username mapping strategy in Hadoop.
...
Contributed by Bolke de Bruin
2019-01-04 17:54:15 -05:00
Inigo Goiri
f4e18242bd
HADOOP-16028. Fix NetworkTopology chooseRandom function to support excluded nodes. Contributed by Sihai Ke.
2019-01-04 09:56:58 -08:00
Sunil G
51427cbdfb
HADOOP-15997. KMS client uses wrong UGI after HADOOP-14445. Contributed by Wei-Chiu Chuang.
2019-01-04 21:49:06 +05:30
Pranay Singh
ecdeaa7e6a
HDFS-14084. Need for more stats in DFSClient. Contributed by Pranay Singh.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-01-03 09:56:07 -08:00
Akira Ajisaka
57198bc511
HADOOP-15962. The buffer size is small when unpacking tar archives. Contributed by BELUGA BEHR.
2018-12-28 15:38:47 +09:00
Konstantin V Shvachko
bd2a59e508
HDFS-14170. [SBN read] Fix checkstyle warnings related to SBN reads. Contributed by Konstantin V Shvachko.
2018-12-24 09:39:20 -08:00
Chao Sun
fa8550337d
HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
Erik Krogen
ef3e1929dd
HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
Chen Liang
cae279610b
HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng.
2018-12-24 09:34:01 -08:00
Erik Krogen
b73fb70f97
HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko.
2018-12-24 09:34:01 -08:00
Chao Sun
cbc2f932eb
HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
Chao Sun
8c49135078
HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun.
2018-12-24 09:34:01 -08:00
Chen Liang
a65bb97f5d
HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen.
2018-12-24 09:34:00 -08:00
Chen Liang
d2e85b0b6d
HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
Chen Liang
66ec3a67e8
HDFS-13767. Add msync server implementation. Contributed by Chen Liang.
2018-12-24 09:34:00 -08:00
Plamen Jeliazkov
e27708c2da
HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov.
2018-12-24 09:34:00 -08:00
Chao Sun
64b7cf59bd
HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
Erik Krogen
151c8ddbe4
HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen.
2018-12-24 09:33:59 -08:00
Plamen Jeliazkov
e880660a20
HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
Erik Krogen
091ad974cd
HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun.
2018-12-24 09:33:59 -08:00
Erik Krogen
9bf0696c73
HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
Plamen Jeliazkov
c9d73437e8
HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov.
2018-12-24 09:33:59 -08:00
Yiqun Lin
feb2664ac4
HDFS-13946. Log longest FSN write/read lock held stack trace.
2018-12-22 23:09:59 +08:00
Surendra Singh Lilhore
763e96ea2b
HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan.
2018-12-22 12:22:24 +05:30
Jason Lowe
3961690037
HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne
2018-12-20 09:29:48 -06:00
Takanobu Asanuma
d963575918
HDFS-13661. Ls command with e option fails when the filesystem is not HDFS.
2018-12-20 10:03:33 +09:00
Surendra Singh Lilhore
788e7473a4
HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena.
2018-12-17 11:24:57 +05:30
Gabor Bota
1ea29b7385
HADOOP-16000. Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-12-15 10:53:52 +09:00
Giovanni Matteo Fumarola
76efeacd5f
HADOOP-15995. Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping. Contributed by Lukas Majercak.
2018-12-12 11:10:24 -08:00
Steve Loughran
202926ac33
HADOOP-15808. Harden Token service loader use.
...
Contributed by Steve Loughran.
2018-12-11 17:33:49 +00:00
Giovanni Matteo Fumarola
aa89492f29
Revert "HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr."
...
This reverts commit fb9deed41d
.
2018-12-04 12:57:28 -08:00
Steve Loughran
db2d8b01c6
HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset.
...
Contributed by Steve Loughran
2018-12-04 15:35:43 +00:00
Giovanni Matteo Fumarola
fb9deed41d
HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr.
2018-12-03 13:25:02 -08:00
Giovanni Matteo Fumarola
c9a3aa64dc
HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak.
2018-12-03 12:10:05 -08:00
Eric Yang
38ea3814bd
HADOOP-15922. Revert patch 004.
2018-11-30 12:49:46 -05:00
Akira Ajisaka
c9bfca217f
MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin.
2018-11-30 13:13:23 +09:00
Steve Loughran
d0edd37269
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple"
...
This reverts commit 829a2e4d27
.
2018-11-29 17:52:11 +00:00
Ewan Higgs
c1d24f8483
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
...
Contributed by Ewan Higgs and Steve Loughran.
2018-11-29 15:12:17 +00:00
Eric Yang
a9d96948ed
HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter.
...
Contributed by He Xiaoqiao
2018-11-26 13:49:19 -05:00
Vinayakumar B
23b441c225
HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena.
2018-11-26 18:11:01 +05:30
Márton Elek
6934a65402
HADOOP-15358. SFTPConnectionPool connections leakage. Contributed by Mikhail Pryakhin.
2018-11-23 09:48:26 +01:00
Akira Ajisaka
1a00b4e325
Add 2.9.2 release notes and changes documents.
2018-11-20 14:25:18 +09:00
Inigo Goiri
460a94a10f
HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang.
2018-11-15 10:58:57 -08:00
Surendra Singh Lilhore
75291e6d53
HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore
2018-11-13 20:22:58 +05:30
Steve Loughran
3c9d97b8f7
HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds.
...
Contributed by LiXin Ge.
2018-11-12 12:58:05 +00:00
Giovanni Matteo Fumarola
89b49167a5
HADOOP-15903. Allow HttpServer2 to discover resources in /static when symlinks are used. Contributed by Inigo Goiri.
2018-11-08 14:52:24 -08:00
Jason Lowe
0b6625a973
MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan
2018-11-07 08:20:49 -06:00
Jason Lowe
ab6aa4c726
YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg
2018-11-06 08:40:59 -06:00
Lars Francke
cb8d679c95
HADOOP-15687. Credentials class should allow access to aliases.
...
Author: Lars Francke <lars.francke@gmail.com>
2018-11-03 16:21:29 +00:00
Giovanni Matteo Fumarola
44e37b4fd9
HADOOP-15885. Add base64 (urlString) support to DTUtil. Contributed by Inigo Goiri.
2018-11-02 10:54:12 -07:00
Steve Loughran
62d98ca92a
HADOOP-15855. Review hadoop credential doc, including object store details.
...
Contributed by Steve Loughran.
2018-10-30 15:58:04 +00:00
Wei-Chiu Chuang
63e7134df6
Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao."
...
This reverts commit fb2b72e6fc
.
2018-10-28 17:04:59 -07:00
Hrishikesh Gadre
bfb9adc2b9
HADOOP-9567. Provide auto-renewal for keytab based logins. Contributed by Hrishikesh Gadre, Gary Helmling and Harsh J.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-27 08:59:47 -07:00
Wei-Chiu Chuang
fb2b72e6fc
HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao.
2018-10-25 09:33:31 -07:00
Akira Ajisaka
8f97d6f2cd
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe.
2018-10-24 13:49:28 +09:00
Haibo Chen
823bb5dda8
MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter)
2018-10-23 15:28:13 -07:00
Chen Liang
635786a511
HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang.
2018-10-23 14:53:45 -07:00
Inigo Goiri
dd268a64d3
Revert "HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR."
...
This reverts commit 00254d7b8c
.
2018-10-23 09:23:03 -07:00
Steve Loughran
52cb766ad0
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility.
...
Contributed by Wei-Chiu Chuang.
2018-10-22 11:21:12 +01:00
Eric Yang
e2a9fa8448
HADOOP-15821. Move YARN Registry to Hadoop Registry.
...
Contributed by Íñigo Goiri
2018-10-19 19:46:48 -04:00
Inigo Goiri
00254d7b8c
HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR.
2018-10-19 13:57:43 -07:00
Jitendra Pandey
d93d515af5
HDDS-651. Rename o3 to o3fs for Filesystem.
2018-10-17 14:19:17 -07:00
Jason Lowe
9abda83947
HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe
2018-10-17 14:38:42 -05:00