big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,374 Commits 2 Branches 2 Tags 264 MiB
d37dc5d1b8
Commit Graph

2565 Commits

Author SHA1 Message Date
Kihwal Lee
9422515239 HADOOP-13442. Optimize UGI group lookups. Contributed by Daryn Sharp. 2016-08-04 10:45:55 -05:00
Xiao Chen
05db64913d HADOOP-13443. KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default. Contributed by Anthony Young-Garner. 2016-08-04 15:57:34 +08:00
Wei-Chiu Chuang
a1f6564d31 HADOOP-13458. LoadBalancingKMSClientProvider#doOp should log IOException stacktrace. Contributed by Chen Liang. 2016-08-03 14:19:47 -07:00
Jason Lowe
c82745432a HADOOP-10980. TestActiveStandbyElector fails occasionally in trunk. Contributed by Eric Badger 2016-08-03 20:17:30 +00:00
Kihwal Lee
580a833496 HADOOP-13483. Optimize IPC server protobuf decoding. Contributed by Daryn Sharp. 2016-08-03 13:22:22 -05:00
Kihwal Lee
2d8227605f HADOOP-13426. More efficiently build IPC responses. Contributed by Daryn Sharp. 2016-08-03 09:31:49 -05:00
Arpit Agarwal
d28c2d9f55 HDFS-9353. Code and comment mismatch in JavaKeyStoreProvider. (Andras Bokor) 2016-08-02 15:17:45 -07:00
Arpit Agarwal
954465e7ba HADOOP-13434. Add bash quoting to Shell class. (Owen O'Malley) 2016-08-02 13:40:33 -07:00
Chris Nauroth
0458a2af6e HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin. 2016-08-02 12:43:30 -07:00
Kihwal Lee
b3018e73cc HADOOP-13429. Dispose of unnecessary SASL servers. Contributed by Daryn Sharp. 2016-08-02 10:40:28 -05:00
Akira Ajisaka
770b5eb2db HADOOP-13444. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-01 17:35:59 +09:00
Kai Zheng
34ccaa8367 HADOOP-11540. Raw Reed-Solomon coder using Intel ISA-L library. Contributed by Kai Zheng 2016-08-01 06:34:26 +08:00
Akira Ajisaka
3d191cc152 HADOOP-13440. FileContext does not react on changing umask via configuration. 2016-07-31 11:29:15 +09:00
Xiao Chen
8ebf2e95d2 HADOOP-13381. KMS clients should use KMS Delegation Tokens from current UGI. Contributed by Xiao Chen. 2016-07-28 18:24:41 -07:00
Xiao Chen
d9aae22fdf HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 2016-07-28 16:29:40 -07:00
Lei Xu
a3d0cba810 HDFS-10689. Hdfs dfs chmod should reset sticky bit permission when the bit is omitted in the octal mode. (Manoj Govindassamy via Lei Xu) 2016-07-28 13:52:14 -07:00
Chris Nauroth
255ea45e50 HADOOP-13422. ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process. Contributed by Sergey Shelukhin. 2016-07-26 15:33:20 -07:00
Chris Douglas
22a9a6b0d4 HADOOP-13272. ViewFileSystem should support storage policy related API. Contributed by Peter Shi 2016-07-22 18:08:12 -07:00
Chris Nauroth
5fda66e4a3 HADOOP-13207. Specify FileSystem listStatus, listFiles and RemoteIterator. Contributed by Steve Loughran. 2016-07-22 15:38:18 -07:00
Chris Nauroth
43cf6b101d HADOOP-13240. TestAclCommands.testSetfaclValidations fail. Contributed by John Zhuge. 2016-07-21 14:12:31 -07:00
Steve Loughran
37362c2f92 HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan) 2016-07-20 13:42:51 +01:00
Akira Ajisaka
dc065dd64c HADOOP-12991. Conflicting default ports in DelegateToFileSystem. Contributed by Kai Sasaki. 2016-07-19 16:05:48 -07:00
Akira Ajisaka
92fe2db540 HADOOP-13301. Millisecond timestamp for FsShell console log and MapReduce jobsummary log. Contributed by John Zhuge. 2016-07-18 15:45:44 -07:00
Akira Ajisaka
c2bcffb34e HADOOP-13202. Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes. Contributed by Kai Sasaki. 2016-07-18 11:48:23 -07:00
Steve Loughran
76be82bc04 HADOOP-13073 RawLocalFileSystem does not react on changing umask. Contributed by Andras Bokor 2016-07-18 14:38:35 +01:00
Robert Kanter
442162048a HADOOP-13254. Create framework for configurable disk checkers (yufeigu via rkanter) 2016-07-15 13:41:26 -07:00
Steve Loughran
be26c1b9b0 Revert "HADOOP-13073 RawLocalFileSystem does not react on changing umask. Contributed by Andras Bokor" 
This reverts commit 45af585e7a.
 2016-07-14 10:30:36 +01:00
Steve Loughran
45af585e7a HADOOP-13073 RawLocalFileSystem does not react on changing umask. Contributed by Andras Bokor 2016-07-14 10:29:29 +01:00
Zhe Zhang
728bf7f698 HADOOP-13290. Appropriate use of generics in FairCallQueue. Contributed by Jonathan Hung. 2016-07-13 16:37:40 -07:00
Akira Ajisaka
505ce88437 HADOOP-13366. Fix dead link in o.a.h.fs.CommonConfigurationKeysPublic javadoc. Contributed by Rakesh R. 2016-07-13 13:02:20 -07:00
Tsuyoshi Ozawa
77ffe76212 HADOOP-11361. Fix a race condition in MetricsSourceAdapter.updateJmxCache. Contributed by Vinayakumar B, Yongjun Zhang, and Brahma Reddy Battula. (ozawa) 2016-07-13 21:28:04 +09:00
Lei Xu
a290a98b6a HADOOP-13315. FileContext#umask is not initialized properly. (John Zhuge via lei) 2016-07-12 16:49:32 -07:00
Jitendra Pandey
ef30bf3c3f HADOOP-13352. Make X-FRAME-OPTIONS configurable in HttpServer2. Contributed by Anu Engineer. 2016-07-08 14:17:14 -07:00
Jitendra Pandey
6e597600f7 HADOOP-13283. Support reset operation for new global storage statistics and per FS storage stats. Contributed by Mingliang Liu. 2016-07-07 14:58:19 -07:00
Allen Wittenauer
a0035661c1 HADOOP-13349. HADOOP_CLASSPATH vs HADOOP_USER_CLASSPATH (aw) 2016-07-07 14:15:34 -07:00
Akira Ajisaka
d1626a7b37 MAPREDUCE-6717. Remove deprecated StringUtils.getFormattedTimeWithDiff. Contributed by Shen Yinjie. 2016-07-07 11:18:43 -07:00
Jian He
d792a90206 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 2016-07-05 18:05:16 -07:00
Jitendra Pandey
aa42c7a6dd HADOOP-13305. Define common statistics names across schemes. Contributed by Mingliang Liu. 2016-07-01 15:34:03 -07:00
Steve Loughran
96fa0f848b HADOOP-12709 Cut s3:// from trunk. Contributed by Mingliang Liu. 2016-06-29 16:04:50 +01:00
Allen Wittenauer
b3649adf6a HDFS-9852. hdfs dfs -setfacl error message is misleading (Wei-Chiu Chuang via aw) 2016-06-28 20:47:05 -07:00
Andrew Wang
77031a9c37 HADOOP-12864. Remove bin/rcc script. Contributed by Allen Wittenauer. 2016-06-28 14:25:02 -07:00
Allen Wittenauer
23c3ff85a9 HADOOP-13209. replace slaves with workers (John Smith via aw) 2016-06-28 05:53:03 -07:00
Andrew Wang
7d2070493e HADOOP-13316. Enforce Kerberos authentication for required ops in DelegationTokenAuthenticator. Contributed by Xiao Chen. 2016-06-27 18:46:33 -07:00
Andrew Wang
771f798edf HADOOP-13251. Authenticate with Kerberos credentials when renewing KMS delegation token. Contributed by Xiao Chen. 2016-06-27 18:20:56 -07:00
Arpit Agarwal
9683eab0e1 HADOOP-13263. Reload cached groups in background after expiry. (Contributed bt Stephen O'Donnell) 2016-06-27 09:36:05 -07:00
Tsz-Wo Nicholas Sze
d328e66706 HADOOP-13227. AsyncCallHandler should use an event driven architecture to handle async calls. 2016-06-25 15:45:16 +08:00
Sangjin Lee
bf74dbf80d HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) 2016-06-24 14:44:15 -07:00
Colin Patrick Mccabe
975786492c Fix accidental revert of HADOOP-13072: WindowsGetSpaceUsed constructor should be public (Vinayakumar B via cmccabe) 2016-06-24 09:46:36 -07:00
Zhe Zhang
0b9edf6e0f HADOOP-13019. Implement ErasureCodec for HitchHiker XOR coding. Contributed by Kai Sasaki. 2016-06-23 15:52:51 -07:00
Andrew Wang
e98c0c7a1c Revert "HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen." 
This reverts commit 35356de1ba.
 2016-06-23 11:08:02 -07:00
Steve Loughran
4ee3543625 HADOOP-13203 S3A: Support fadvise "random" mode for high performance readPositioned() reads. Contributed by Rajesh Balamohan and stevel. 2016-06-22 15:45:25 +01:00
Akira Ajisaka
e15cd43369 HADOOP-13296. Cleanup javadoc for Path. Contributed by Daniel Templeton. 2016-06-21 20:16:22 +09:00
Colin Patrick Mccabe
8c1f81d4bf HADOOP-13288. Guard null stats key in FileSystemStorageStatistics (Mingliang Liu via Colin P. McCabe) 2016-06-20 14:25:07 -07:00
Colin Patrick Mccabe
5370a6ffae HADOOP-13280. FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps (Mingliang Liu via cmccabe) 2016-06-20 10:46:18 -07:00
Akira Ajisaka
fc6b50cc57 HADOOP-13192. org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly. Contributed by binde. 2016-06-20 17:07:26 +09:00
Arpit Agarwal
0319d73c3b HDFS-10437. ReconfigurationProtocol not covered by HDFSPolicyProvider. (Arpit Agarwal) 2016-06-18 16:16:54 -07:00
Chris Nauroth
5f6bc65bb3 Revert "HADOOP-12718. Incorrect error message by fs -put local dir without permission. (John Zhuge via Yongjun Zhang)" 
This reverts commit 97056c3355.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-06-17 23:50:22 -07:00
Xiaoyu Yao
0761379fe4 HADOOP-13285. DecayRpcScheduler MXBean should only report decayed CallVolumeSummary. Contributed by Xiaoyu Yao. 2016-06-17 15:25:14 -07:00
Akira Ajisaka
09e82acaf9 HADOOP-12943. Add -w -r options in dfs -test command. Contributed by Weiwei Yang. 2016-06-17 16:20:38 +09:00
Akira Ajisaka
51d497fa93 MAPREDUCE-6542. HistoryViewer uses SimpleDateFormat, but SimpleDateFormat is not threadsafe. Contributed by zhangyubiao. 2016-06-17 11:03:54 +09:00
Konstantin V Shvachko
a2a5cb60b0 HADOOP-13189. FairCallQueue makes callQueue larger than the configured capacity. Contributed by Vinitha Gankidi. 2016-06-16 18:20:49 -07:00
Colin Patrick Mccabe
bf780406f2 HADOOP-12975. Add jitter to CachingGetSpaceUsed's thread (Elliott Clark via Colin P. McCabe) 2016-06-16 16:48:05 -07:00
Xiaoyu Yao
b1674caa40 HADOOP-13255. KMSClientProvider should check and renew tgt when doing delegation token operations. Contributed by Xiao Chen. 2016-06-16 15:22:00 -07:00
Chris Nauroth
127d2c7281 HADOOP-13241. document s3a better. Contributed by Steve Loughran. 2016-06-16 11:18:02 -07:00
Vinayakumar B
cc6c265171 HDFS-10256. Use GenericTestUtils.getTestDir method in tests for temporary directories (Contributed by Vinayakumar B) 2016-06-16 16:47:06 +05:30
Jitendra Pandey
6f0aa75121 HADOOP-12291. Add support for nested groups in LdapGroupsMapping. Contributed by Esther Kundin. 2016-06-15 11:41:49 -07:00
Vinayakumar B
8e8cb4c6ca HADOOP-13179. GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe. (Contributed by hongbin ma) 2016-06-14 16:01:26 +05:30
Masatake Iwasaki
20b13d109a HADOOP-13244. o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in consturctor. Contributed by Kai Sasaki. 2016-06-14 19:20:46 +09:00
Akira Ajisaka
e3ba9ad3f1 HADOOP-13270. BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks. Contributed by Kai Sasaki. 2016-06-14 10:18:41 +09:00
Andrew Wang
0accc3306d HADOOP-13079. Add -q option to Ls to print ? instead of non-printable characters. Contributed by John Zhuge. 2016-06-13 11:43:46 -07:00
Jing Zhao
0bbb4ddd79 HADOOP-13249. RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep. Contributed by Zhihai Xu. 2016-06-10 10:38:13 -07:00
Chris Nauroth
9581fb715c HADOOP-12666. Support Microsoft Azure Data Lake - as a file system in Hadoop. Contributed by Vishwajeet Dusane. 2016-06-09 14:33:31 -07:00
Steve Loughran
31ffaf76f2 HADOOP-12537 S3A to support Amazon STS temporary credentials. Contributed by Sean Mackrory. 2016-06-09 21:00:47 +01:00
Steve Loughran
656c460c0e HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials. Contributed by Chris Nauroth 2016-06-09 17:28:49 +01:00
Junping Du
c14c1b298e HADOOP-10048. LocalDirAllocator should avoid holding locks while accessing the filesystem. Contributed by Jason Lowe. 2016-06-07 09:18:58 -07:00
Tsz-Wo Nicholas Sze
35f255b03b Revert "Revert "HADOOP-13226 Support async call retry and failover."" 
This reverts commit 5360da8bd9.
 2016-06-06 16:31:43 +08:00
Tsz-Wo Nicholas Sze
574dcd34c0 Revert "Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls."" 
This reverts commit e4450d47f1.
 2016-06-06 16:31:23 +08:00
Tsz-Wo Nicholas Sze
aa20fa150d Revert "Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou"" 
This reverts commit 4d36b221a2.
 2016-06-06 16:28:47 +08:00
Tsz-Wo Nicholas Sze
eded3d109e Revert "Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou"" 
This reverts commit 106234d873.
 2016-06-06 16:28:21 +08:00
Andrew Wang
106234d873 Revert "HDFS-10224. Implement asynchronous rename for DistributedFileSystem. Contributed by Xiaobing Zhou" 
This reverts commit fc94810d3f.
 2016-06-03 18:09:55 -07:00
Andrew Wang
4d36b221a2 Revert "HADOOP-12957. Limit the number of outstanding async calls. Contributed by Xiaobing Zhou" 
This reverts commit 1b9f18623a.
 2016-06-03 18:09:18 -07:00
Andrew Wang
e4450d47f1 Revert "HADOOP-13168. Support Future.get with timeout in ipc async calls." 
This reverts commit 42c22f7e3d.
 2016-06-03 18:09:16 -07:00
Andrew Wang
5360da8bd9 Revert "HADOOP-13226 Support async call retry and failover." 
This reverts commit 83f2f78c11.
 2016-06-03 18:09:09 -07:00
Andrew Wang
713cb71820 HADOOP-13155. Implement TokenRenewer to renew and cancel delegation tokens in KMS. Contributed by Xiao Chen. 2016-06-03 16:48:54 -07:00
Chris Nauroth
d82bc85018 HADOOP-13105. Support timeouts in LDAP queries in LdapGroupsMapping. Contributed by Mingliang Liu. 2016-06-03 16:38:30 -07:00
Allen Wittenauer
78b3a03831 HADOOP-13109. Add ability to edit existing token file via dtutil -alias flag (Matthew Paduano via aw) 2016-06-03 15:34:39 -07:00
Jian He
99cc439e29 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 2016-06-03 11:10:42 -07:00
Akira Ajisaka
99675e00df HADOOP-13232. Typo in exception in ValueQueue.java. Contributed by Jiayi Zhou. 2016-06-02 19:14:53 +09:00
Andrew Wang
35356de1ba HADOOP-13228. Add delegation token to the connection in DelegationTokenAuthenticator. Contributed by Xiao Chen. 2016-06-01 13:13:17 -07:00
Colin Patrick Mccabe
8ceb06e239 HADOOP-13137. TraceAdmin should support Kerberized cluster (Wei-Chiu Chuang via cmccabe) 2016-05-31 17:54:34 -07:00
Tsz-Wo Nicholas Sze
83f2f78c11 HADOOP-13226 Support async call retry and failover. 2016-05-31 16:30:11 -07:00
Andrew Wang
bca31fe276 HADOOP-13132. Handle ClassCastException on AuthenticationException in LoadBalancingKMSClientProvider. Contributed by Wei-Chiu Chuang. 2016-05-31 09:35:24 -07:00
Kai Zheng
93d8a7f2a2 Revert "HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng" 
This reverts commit a6c79f92d5.
 2016-06-01 08:41:15 +08:00
Xiaoyu Yao
4ca8859583 HADOOP-13197. Add non-decayed call metrics for DecayRpcScheduler. Contributed by Xiaoyu Yao. 2016-05-27 18:07:44 -07:00
Kai Zheng
916140604f HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li 2016-05-28 14:23:39 +08:00
Yongjun Zhang
34cc21f6d1 HADOOP-12847. hadoop daemonlog should support https and SPNEGO for Kerberized cluster. (Wei-Chiu Chuang via Yongjun Zhang) 2016-05-26 22:43:45 -07:00
Kai Zheng
77202fa103 HADOOP-13010. Refactor raw erasure coders. Contributed by Kai Zheng 2016-05-27 13:23:34 +08:00
Allen Wittenauer
3c83cee118 HADOOP-12925. Checks for SPARC architecture need to include 64-bit SPARC (Alan Burlison via aw) 2016-05-25 15:28:02 -07:00
Kai Zheng
a6c79f92d5 HADOOP-12579. Deprecate and remove WriteableRPCEngine. Contributed by Kai Zheng 2016-05-27 04:29:42 +08:00
Karthik Kambatla
57c31a3fef HDFS-9782. RollingFileSystemSink should have configurable roll interval. (Daniel Templeton via kasha) 2016-05-24 10:37:00 -07:00