Commit Graph

4390 Commits

Author SHA1 Message Date
Arpit Agarwal
a6fdeb8a87 HADOOP-13963. /bin/bash is hard coded in some of the scripts. Contributed by Ajay Yadav. 2017-08-04 10:40:52 -07:00
wenxinhe
691bf5ec5d
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement.
This closes #257

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-08-04 14:35:57 +09:00
Andrew Wang
f4c6b00a9f HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:44:51 -07:00
Chris Douglas
12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Sean Mackrory
1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Steve Loughran
5e4434f628
HADOOP-14709. Fix checkstyle warnings in ContractTestUtils.
Contributed by Thomas Marquardt.

(cherry picked from commit cb4679261097e729b433a523e3a65779fdc83841)
2017-08-02 12:48:06 +01:00
Masatake Iwasaki
3a0b84d9c9 HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu. 2017-08-02 06:06:25 +00:00
Junping Du
cf6794ebe9 HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 2017-08-01 17:13:02 -07:00
Lei Xu
02cd71ba9d HADOOP-14495. Add set options interface to FSDataOutputStreamBuilder. (Lei (Eddy) Xu) 2017-08-01 16:00:53 -07:00
Andrew Wang
a11c230236 HADOOP-14701. Configuration can log misleading warnings about an attempt to override final parameter. Contributed by Andrew Sherman. 2017-08-01 15:13:29 -07:00
Steve Loughran
778d4edd9a
HADOOP-14660 wasb: improve throughput by 34% when account limit exceeded.
Contributed by Thomas Marquardt.
2017-08-01 21:36:28 +01:00
Arpit Agarwal
72eba25e09 HDFS-14676. Wrong default value for fs.df.interval. Contributed by xiangguang zheng. 2017-08-01 13:31:44 -07:00
Lei Xu
9586b0e24f HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 2017-07-31 20:12:40 -07:00
Andras Bokor
abbf4129a2
HADOOP-14343. Wrong pid file name in error message when starting secure daemon
Signed-off-by: Allen Wittenauer <aw@apache.org>
2017-07-31 20:03:43 -07:00
Akira Ajisaka
f14be0d241
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang. 2017-07-31 14:08:30 +09:00
Ravi Prakash
746189ad8c HADOOP-14229. hadoop.security.auth_to_local example is incorrect in the documentation. Contributed by Andras Bokor. 2017-07-28 11:43:36 -07:00
Yongjun Zhang
c6330f22a5 HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 2017-07-27 16:48:24 -07:00
Kihwal Lee
27a1a5fde9 HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp. 2017-07-26 13:12:39 -05:00
Brahma Reddy Battula
6d983cca52 HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-25 23:51:53 +08:00
Akira Ajisaka
cca51e916b
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor. 2017-07-25 15:24:56 +09:00
Jitendra Pandey
f2921e51f0 HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov. 2017-07-24 23:01:01 -07:00
Ravi Prakash
94ca52ae9e HADOOP-14597. Native compilation broken with OpenSSL-1.1.0. Contributed by Ravi Prakash. 2017-07-24 16:01:45 -07:00
Jason Lowe
24853bf32a YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 2017-07-24 10:19:20 -05:00
Wei-Chiu Chuang
465c213439 HDFS-12094. Log torrent when none isa-l EC is used. Contributed by LiXin Ge. 2017-07-21 12:25:03 -07:00
Kihwal Lee
b3269f7cc1 HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 2017-07-20 16:12:21 -05:00
Daniel Templeton
c21c260392 HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals() 2017-07-19 13:58:55 -07:00
Jason Lowe
df180259b0 HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton 2017-07-19 09:41:22 -05:00
Jitendra Pandey
2843c688bc HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg. 2017-07-19 00:13:06 -07:00
Jason Lowe
5aa2bf231f HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton 2017-07-18 16:23:41 -05:00
Akira Ajisaka
ccaf036662
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common. Contributed by Wenxin He. 2017-07-18 13:32:37 +09:00
Anu Engineer
02b141ac60 HDFS-11786. Add support to make copyFromLocal multi threaded. Contributed by Mukul Kumar Singh. 2017-07-16 10:59:34 -07:00
Xiao Chen
0a6d5c0cf1 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:22:53 -07:00
Masatake Iwasaki
b61ab8573e HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor. 2017-07-13 21:41:43 +09:00
Akira Ajisaka
9144fd9e9b
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 2017-07-12 11:35:50 +09:00
Steve Loughran
d670c3a4da
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
Contributed by Thomas Marquardt
2017-07-11 21:34:27 +01:00
Yiqun Lin
3a7f02b815 HDFS-12085. Reconfigure namenode heartbeat interval fails if the interval was set with time unit. Contributed by Weiwei Yang. 2017-07-11 20:40:11 +08:00
wenxinhe
fce7951014
HADOOP-14638. Replace commons-logging APIs with slf4j in StreamPumper.
This closes #247

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-11 13:30:13 +09:00
Jitendra Pandey
f1efa14fc6 HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 2017-07-10 17:48:27 -07:00
Steve Loughran
09653ea098
HADOOP-14634. Remove jline from main Hadoop pom.xml.
Contributed by Ray Chiang.
2017-07-10 11:53:13 +01:00
Andrew Wang
f10864a820 Add release notes, changes, jdiff for 3.0.0-alpha4 2017-07-07 11:01:59 -07:00
Akira Ajisaka
7cd095272c
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-08 02:54:24 +09:00
Akira Ajisaka
8fc5dcc2a1
Revert "HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He."
This reverts commit 82cb2a6497.
2017-07-08 02:53:18 +09:00
Wei-Chiu Chuang
8153fe2bd3 HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah. 2017-07-07 06:13:10 -07:00
Akira Ajisaka
82cb2a6497
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-07 15:00:47 +09:00
Vinayakumar B
a180ba4081 HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore. 2017-07-05 16:05:18 +05:30
Steve Loughran
b17e655b70 HADOOP-14617. Add ReflectionUtils.logThreadInfo that accept slf4j logger API.
Contributed by Wenxin He.
2017-07-04 11:41:07 +01:00
Wenxin He
f2aba1da30
HADOOP-14571. Deprecate public APIs relate to log4j1
This closes #244

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-04 18:55:20 +09:00
Steve Loughran
1aaa7f1eac
HADOOP-14615. Add ServiceOperations.stopQuietly that accept slf4j logger API.
Contributed by Wenxin He.
2017-07-04 10:48:02 +01:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Daniel Templeton
5a75f73893 HADOOP-14611. NetworkTopology.DEFAULT_HOST_LEVEL is unused
(Contributed by Chen Liang via Daniel Templeton)
2017-06-29 12:28:43 -07:00
Kai Zheng
ea1da39b19 HADOOP-14479. Erasurecode testcase failures with native enabled. Contributed by Sammi Chen 2017-06-29 11:26:36 +08:00
Lei Xu
d6df0fdbbd Add -E option in 'ls' to list erasure coding policy of each file and directory if applicable. Contributed by luhuichun via lei. 2017-06-28 13:47:23 -07:00
Kai Zheng
379f19a2c7 HDFS-11943. [Erasure coding] Warn log frequently print to screen in doEncode/doDecode functions. Contributed by liaoyuxiangqin 2017-06-26 16:26:09 +08:00
Akira Ajisaka
a594f96fb1
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 2017-06-26 16:41:00 +09:00
Arpit Agarwal
0111711d8b HADOOP-14543. ZKFC should use getAversion() while setting the zkacl. Contributed by Brahma Reddy Battula. 2017-06-23 16:50:47 -07:00
Steve Loughran
e5db9af8a0
HADOOP-14568. GenericTestUtils#waitFor missing parameter verification.
Contributed by Yiqun Lin
2017-06-23 14:39:58 +01:00
Akira Ajisaka
092ebdf885
HADOOP-12940. Fix warnings from Spotbugs in hadoop-common. 2017-06-23 10:28:58 +09:00
Arpit Agarwal
8ceaac1877 HDFS-12010. TestCopyPreserveFlag fails consistently because of mismatch in access time. Contributed by Mukul Kumar Singh. 2017-06-22 15:42:50 -07:00
Vinayakumar B
8dbd53ef9f HDFS-11067. DFS#listStatusIterator(..) should throw FileNotFoundException if the directory deleted before fetching next batch of entries. Contributed by Vinayakumar B. 2017-06-22 17:37:08 +05:30
Akira Ajisaka
b64951905e
HADOOP-14542. Add IOUtils.cleanupWithLogger that accepts slf4j logger API. Contributed by Chen Liang. 2017-06-22 17:42:59 +09:00
Akira Ajisaka
1a598479a9
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-21 11:19:48 +09:00
Yongjun Zhang
5157f6c46e HADOOP-14429. FTPFileSystem#getFsAction always returns FsAction.NONE. (Hongyuan Li via Yongjun Zhang) 2017-06-20 16:51:13 -07:00
Lei Xu
45ff4d38e6 HDFS-11647. Add -E option in hdfs "count" command to show erasure policy summarization. Contributed by luhuichun. 2017-06-20 11:55:09 -07:00
Brahma Reddy Battula
2e9daa2e27 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 2017-06-20 20:44:31 +08:00
Akira Ajisaka
300804595e
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong. 2017-06-19 19:39:36 +09:00
Akira Ajisaka
7ade5124b8
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu. 2017-06-19 19:09:18 +09:00
Masatake Iwasaki
75043d3429 HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge. 2017-06-19 09:16:45 +09:00
Lei Xu
6460df21a0 HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 2017-06-16 17:24:00 -07:00
Xiao Chen
4ebc23ba7b HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:33 -07:00
Steve Loughran
942ecbbc98
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0.
Contributed by Hongyuan Li.
2017-06-16 09:45:22 +01:00
Akira Ajisaka
e855cc4ed4
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common. 2017-06-16 13:17:10 +09:00
Lei Xu
5e7cfdca7b HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 2017-06-15 10:59:24 -07:00
Xiao Chen
ef8edab930 HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev. 2017-06-15 10:40:59 -07:00
Steve Loughran
d780a67864
HADOOP-14506. Add create() contract test that verifies ancestor dir creation.
Contributed by Sean Mackrory.
2017-06-15 14:49:34 +01:00
Arpit Agarwal
8633ef8e10 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-12 20:43:43 -07:00
Jonathan Eagles
a81916ea89 HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:07:53 -05:00
Mingliang Liu
a2121cb0d9 HADOOP-14465. LdapGroupsMapping - support user and group search base. Contributed by Shwetha G S and Mingliang Liu 2017-06-09 14:55:07 -07:00
Xiao Chen
0202480742 HADOOP-13174. Add more debug logs for delegation tokens and authentication. 2017-06-08 21:34:15 -07:00
John Zhuge
5672ae7b37 HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge. 2017-06-07 15:52:52 -07:00
Carlo Curino
b65100c14b YARN-6547. Enhance SLS-based tests leveraging invariant checker. 2017-06-06 17:19:15 -07:00
Brahma Reddy Battula
19ef3a81f8 HADOOP-14485. Redundant 'final' modifier in try-with-resources statement. Contributed by wenxin he. 2017-06-06 23:11:47 +08:00
Kihwal Lee
855e0477b1 HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp. 2017-06-06 08:35:12 -05:00
Brahma Reddy Battula
66c6fd8314 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 2017-06-06 12:31:40 +08:00
Mingliang Liu
6aeda55bb8 HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 2017-06-05 11:26:56 -07:00
Brahma Reddy Battula
abdd609e51 HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 2017-06-06 00:21:03 +08:00
Andrew Wang
7101477e47 HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 2017-06-01 15:20:18 -07:00
Kihwal Lee
1543d0f5be HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 10:55:03 -05:00
Brahma Reddy Battula
62857be211 HADOOP-14456. Modifier 'static' is redundant for inner enums. Contributed by ZhangBing Lin. 2017-05-31 01:07:58 +08:00
Arpit Agarwal
bd6a2172e0 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:40 -07:00
Andrew Wang
2cd612ba8e Update 3.0.0-alpha3 changes, release notes, jdiff. 2017-05-26 14:14:38 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Steve Loughran
8bf0e2d6b3
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's
getFileStatus method is always 0.
Contributed by Hongyuan Li.
2017-05-25 15:19:58 +01:00
Steve Loughran
1ba9704eec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs.
Contributed by Jonathan Eagles
2017-05-25 14:59:33 +01:00
Akira Ajisaka
6a52b5e144
HADOOP-14180. FileSystem contract tests to replace JUnit 3 with 4. Contributed by Xiaobing Zhou and Mingliang Liu. 2017-05-25 15:36:44 +09:00
Brahma Reddy Battula
dcf4559ebd HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 2017-05-25 01:27:13 +08:00
Brahma Reddy Battula
b7a0c0e701 HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin. 2017-05-25 01:03:22 +08:00
Kai Zheng
a62be38a5e HDFS-11793. Allow to enable user defined erasure coding policy. Contributed by Sammi Chen 2017-05-24 18:45:52 +08:00
Brahma Reddy Battula
52661e0912 HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin. 2017-05-23 23:52:42 +08:00
Rakesh Radhakrishnan
1b5451bf05 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2017-05-23 17:03:28 +05:30
Brahma Reddy Battula
2a8fcf0c9a HDFS-11863. Document missing metrics for blocks count in pending IBR. Contributed By Yiqun Lin. 2017-05-23 02:16:25 +08:00