Wilfred Spiegelenburg
2214005c0f
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja
2020-04-08 11:30:03 +10:00
Steve Loughran
20eec95867
HADOOP-16932. distcp copy calls getFileStatus() needlessly and can fail against S3 ( #1936 )
...
Contributed by Steve Loughran.
This strips out all the -p preservation options which have already been
processed when uploading a file before deciding whether or not to query
the far end for the status of the (existing/uploaded) file to see if any
other attributes need changing.
This will avoid 404 caching-related issues in S3, wherein a newly created
file can have a 404 entry in the S3 load balancer's cache from the
probes for the file's existence prior to the upload.
It partially addresses a regression caused by HADOOP-8143,
"Change distcp to have -pb on by default" that causes a resurfacing
of HADOOP-13145, "In DistCp, prevent unnecessary getFileStatus call when
not preserving metadata"
2020-04-07 17:55:55 +01:00
Szilard Nemeth
bffb43b00e
YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja
2020-04-07 17:03:17 +02:00
Szilard Nemeth
45362a9f4c
YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja
2020-04-07 15:51:55 +02:00
Toshihiro Suzuki
c12ddbd1de
HDFS-15249 ThrottledAsyncChecker is not thread-safe. ( #1922 )
2020-04-07 14:38:09 +09:00
Inigo Goiri
0b855b9f35
HDFS-15256. Fix typo in DataXceiverServer#run(). Contributed by Lisheng Sun.
2020-04-06 13:05:33 -07:00
Akira Ajisaka
ab7495dc7d
MAPREDUCE-7269. TestNetworkedJob fails ( #1940 )
2020-04-06 17:57:37 +09:00
Takanobu Asanuma
b89d875f7b
HADOOP-16954. Add -S option in "Count" command to show only Snapshot Counts. Contributed by hemanthboyina.
2020-04-06 11:03:10 +09:00
Akira Ajisaka
e6455cc864
HADOOP-16949. pylint fails in the build environment ( #1926 )
2020-04-04 16:52:04 +09:00
Masatake Iwasaki
8f8be6b92a
HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan.
2020-04-04 07:20:51 +09:00
Akira Ajisaka
1695d8d59c
HDFS-15258. RBF: Mark Router FSCK unstable. ( #1934 )
2020-04-04 00:13:41 +09:00
Ayush Saxena
ff50ecdc10
HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena.
2020-04-03 13:57:02 +05:30
Prabhu Joseph
7a39da3537
YARN-10120. Amendment fix for Java Doc.
...
Contributed by Bilwa S T.
2020-04-03 13:07:41 +05:30
Wanqiang Ji
f8e598f278
MAPREDUCE-7268. Fix TestMapreduceConfigFields ( #1935 )
2020-04-03 10:59:07 +09:00
Prabhu Joseph
25361b077b
YARN-10120. Https Support in Router WebServiceClient.
...
Contributed by Bilwa S T.
2020-04-02 18:11:43 +05:30
Takanobu Asanuma
736659e0e1
HDFS-15252. HttpFS: setWorkingDirectory should not accept invalid paths. Contributed by hemanthboyina.
2020-04-02 19:21:02 +09:00
Ayush Saxena
4a3eb10972
HDFS-15051. RBF: Impose directory level permissions for Mount entries. Contributed by Xiaoqiao He.
2020-04-02 11:06:01 +05:30
He Xiaoqiao
d3b5951572
HDFS-15242. Add metrics for operations hold lock times of FsDatasetImpl. Contributed by Xiaoqiao He.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
2020-04-01 16:36:46 -07:00
Jason Lowe
c613296dc8
MAPREDUCE-7241. FileInputFormat listStatus with less memory footprint. Contributed by Zhihua Deng
2020-04-01 07:46:33 -05:00
Akira Ajisaka
c162648aff
YARN-10202. Fix documentation about NodeAttributes. Contributed by Sen Zhao.
2020-04-01 16:05:47 +09:00
Mehakmeet Singh
c734d247b1
HADOOP-16910 : ABFS Streams to update FileSystem.Statistics counters on IO. ( #1918 ). Contributed by Mehakmeet Singh.
2020-03-31 14:49:09 +02:00
Inigo Goiri
80b877a72f
HDFS-15196. RBF: RouterRpcServer getListing cannot list large dirs correctly. Contributed by Fengnan Li.
2020-03-30 12:29:21 -07:00
Ayush Saxena
960c9ebaea
HDFS-15245. Improve JournalNode web UI. Contributed by Jianfei Jiang.
2020-03-30 02:40:25 +05:30
Brahma Reddy Battula
3eeb2466e9
upate the hadoop.version property in the root pom.xml and hadoop.assemblies.version in hadoop-project/pom.xml (see HADOOP-15369)
2020-03-29 23:44:20 +05:30
Brahma Reddy Battula
8914cf9167
Preparing for 3.4.0 development
2020-03-29 23:24:25 +05:30
Inigo Goiri
f7a94ec0a4
HDFS-15239. Add button to go to the parent directory in the explorer. Contributed by hemanthboyina.
2020-03-29 08:17:02 -07:00
Ayush Saxena
696a663cd0
HDFS-15193. Improving the error message for missing dfs.namenode.rpc-address.$NAMESERVICE. Contributed by Ctest.
2020-03-29 00:01:22 +05:30
Chen Liang
f531a4a487
HDFS-15191. EOF when reading legacy buffer in BlockTokenIdentifier. Contributed by Steven Rand.
2020-03-27 12:55:38 -07:00
Ayush Saxena
cbe71eaddf
HDFS-15238. RBF: NamenodeHeartbeatService caused memory to grow rapidly. Contributed by xuzq.
2020-03-27 13:42:06 +05:30
Steve Loughran
eaaaba12b1
HADOOP-16939 fs.s3a.authoritative.path should support multiple FS URIs ( #1914 )
...
add unit test, new ITest and then fix the issue: different schema, bucket == skip
factored out the underlying logic for unit testing; also moved
maybeAddTrailingSlash to S3AUtils (while retaining/forwarnding existing method
in S3AFS).
tested: london, sole failure is
testListingDelete[auth=true](org.apache.hadoop.fs.s3a.ITestS3GuardOutOfBandOperations)
filed HADOOP-16853
Change-Id: I4b8d0024469551eda0ec70b4968cba4abed405ed
2020-03-26 12:59:11 -06:00
Steve Loughran
745a6c1e69
Revert "HADOOP-16818. ABFS: Combine append+flush calls for blockblob & appendblob"
...
This reverts commit 3612317038
2020-03-26 15:24:37 +00:00
Inigo Goiri
679631b188
HDFS-13470. RBF: Add Browse the Filesystem button to the UI.
2020-03-26 08:23:18 -07:00
RogPodge
50f7f6dfd1
YARN-10210. Add a RMFailoverProxyProvider that does DNS resolution on failover.
2020-03-26 08:14:30 -07:00
Szilard Nemeth
3d5ade1839
YARN-10043. FairOrderingPolicy Improvements. Contributed by Manikandan R
2020-03-26 13:23:43 +01:00
Szilard Nemeth
9a297ff31d
YARN-10197. FS-CS converter: fix emitted ordering policy string and max-am-resource percent value. Contributed by Peter Bacsko
2020-03-26 12:35:10 +01:00
Szilard Nemeth
348685dcb9
YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke
2020-03-26 12:18:15 +01:00
Wei-Chiu Chuang
0fa7bf47df
HDFS-15234. Add a default method body for the INodeAttributeProvider#checkPermissionWithContext API. ( #1909 )
2020-03-25 16:03:26 -07:00
Inigo Goiri
cdcb77a2c5
HDFS-15075. Remove process command timing from BPServiceActor. Contributed by Xiaoqiao He.
2020-03-25 11:30:54 -07:00
Jonathan Hung
6ce189c621
YARN-10200. Add number of containers to RMAppManager summary
2020-03-25 10:41:59 -07:00
Arpit Agarwal
a700803a18
HDFS-15154. Allow only hdfs superusers the ability to assign HDFS storage policies. Contributed by Siddharth Wagle.
...
Change-Id: I32d6dd2837945b8fc026a759aa367c55daefe348
2020-03-25 10:28:30 -07:00
Chao Sun
e3fbdcbc14
HADOOP-16912. Emit per priority RPC queue time and processing time from DecayRpcScheduler. Contributed by Fengnan Li.
2020-03-25 10:21:20 -07:00
RogPodge
2d294bd575
HADOOP-16938. Make non-HA proxy providers pluggable
2020-03-25 08:06:58 -07:00
Sunil G
cdb2107066
YARN-9879. Allow multiple leaf queues with the same name in CapacityScheduler. Contributed by Gergely Pollak.
2020-03-25 16:50:19 +05:30
Nicholas Chammas
25a03bfece
HADOOP-16930. Add hadoop-aws documentation for ProfileCredentialsProvider
...
Contributed by Nicholas Chammas.
2020-03-25 10:39:35 +00:00
Toshihiro Suzuki
d353b30baf
HDFS-15215. The Timestamp for longest write/read lock held log is wrong
2020-03-24 14:50:15 -07:00
Isa Hekmatizadeh
d9c4f1129c
HDFS-15219. DFS Client will stuck when ResponseProcessor.run throw Error ( #1902 ). Contributed by zhengchenyu.
2020-03-24 23:17:22 +05:30
Ayush Saxena
ea87d60493
HDFS-13377. The owner of folder can set quota for his sub folder. Contributed by Yang Yun.
2020-03-24 22:56:09 +05:30
Akira Ajisaka
4454c6d14b
Revert "HADOOP-16054. Update Dockerfile to use Bionic." ( #1911 )
...
This reverts commit 367833cf41
2020-03-24 22:05:33 +09:00
Steve Loughran
28afdce009
Revert ""HADOOP-16910. ABFS Streams to update FileSystem.Statistics counters on IO."
...
This reverts commit e2c7ac71b5
2020-03-24 12:11:18 +00:00
Karthik Palanisamy
5250cd6db3
HDFS-15201 SnapshotCounter hits MaxSnapshotID limit ( #1870 )
2020-03-24 15:15:46 +05:30
