hadoop

Go to file

HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running

This uses the length of the file known at the start of the copy to determine the amount of data to copy.

* If a file is appended to during the copy, the original bytes are copied.
* If a file is truncated during a copy, or the attempt to read the data fails with a truncated stream,
  distcp will now fail. Until now these failures were not detected.

Contributed by Mukund Thakur.

Change-Id: I576a49d951fa48d37a45a7e4c82c47488aa8e884

2019-09-24 11:23:24 +01:00

.github

HADOOP-15184. Add GitHub pull request template. (#1419 )

2019-09-11 11:10:11 +09:00

dev-support

HADOOP-16589. [pb-upgrade] Update docker image to make 3.7.1 protoc as default (#1482 ). Contributed by Vinayakumar B.

2019-09-21 10:07:26 +05:30

hadoop-assemblies

HADOOP-16534. Exclude submarine from hadoop source build. (#1356 )

2019-09-03 17:40:38 +05:30

hadoop-build-tools

HADOOP-16331. Fix ASF License check in pom.xml

2019-05-29 17:25:13 +09:00

hadoop-client-modules

HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432 )

2019-09-20 16:08:30 +05:30

hadoop-cloud-storage-project

HADOOP-16331. Fix ASF License check in pom.xml

2019-05-29 17:25:13 +09:00

hadoop-common-project

HADOOP-16138. hadoop fs mkdir / of nonexistent abfs container raises NPE (#1302 ). Contributed by Gabor Bota.

2019-09-23 13:29:01 +02:00

hadoop-dist

HDFS-14639. [Dynamometer] Remove unnecessary duplicate directory from the distribution. Contributed by Erik Krogen.

2019-07-29 13:50:14 -07:00

hadoop-hdds

HDDS-2159. Fix Race condition in ProfileServlet#pid.

2019-09-23 17:21:19 -07:00

hadoop-hdfs-project

HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar.

2019-09-23 21:22:50 +05:30

hadoop-mapreduce-project

HADOOP-16561. [MAPREDUCE] use protobuf-maven-plugin to generate protobuf classes (#1500 )

2019-09-24 08:52:12 +08:00

hadoop-maven-plugins

HADOOP-16331. Fix ASF License check in pom.xml

2019-05-29 17:25:13 +09:00

hadoop-minicluster

HADOOP-16331. Fix ASF License check in pom.xml

2019-05-29 17:25:13 +09:00

hadoop-ozone

HDDS-2081. Fix TestRatisPipelineProvider#testCreatePipelinesDnExclude. Contributed by Aravindan Vijayan. (#1506 )

2019-09-24 12:59:51 +05:30

hadoop-project

HADOOP-16558. [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes (#1494 ). Contributed by Vinayakumar B.

2019-09-23 12:37:47 +05:30

hadoop-project-dist

Make upstream aware of 3.2.1 release.

2019-09-23 06:20:54 +00:00

hadoop-submarine

SUBMARINE-45. Can't specify queue by using the parameter --queue. Contributed by Ayush Saxena, Zac Zhou.

2019-08-15 13:18:29 +08:00

hadoop-tools

HDFS-13660. DistCp job fails when new data is appended in the file while the DistCp copy job is running

2019-09-24 11:23:24 +01:00

hadoop-yarn-project

HADOOP-16560. [YARN] use protobuf-maven-plugin to generate protobuf classes (#1496 )

2019-09-24 08:50:36 +08:00

licenses

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

licenses-binary

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

.gitattributes

HADOOP-13598. Add eol=lf for unix format files in .gitattributes. Contributed by Yiqun Lin.

2016-09-14 11:14:31 +09:00

.gitignore

HDDS-1115. Provide ozone specific top-level pom.xml.

2019-02-24 14:40:52 -08:00

BUILDING.txt

HADOOP-16557. [pb-upgrade] Upgrade protobuf.version to 3.7.1 (#1432 )

2019-09-20 16:08:30 +05:30

Jenkinsfile

HADOOP-16183. Use latest Yetus to support ozone specific build process

2019-05-02 16:48:30 +02:00

LICENSE-binary

HADOOP-16555. Update commons-compress to 1.19. (#1425 ) Contributed by YiSheng Lien.

2019-09-14 02:11:04 +08:00

LICENSE.txt

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

NOTICE-binary

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

NOTICE.txt

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

pom.ozone.xml

HDDS-2150. Update dependency versions to avoid security vulnerabilities. (#1472 )

2019-09-20 23:21:53 -07:00

pom.xml

HADOOP-15184. Add GitHub pull request template. (#1419 )

2019-09-11 11:10:11 +09:00

README.txt

HADOOP-15958. Revisiting LICENSE and NOTICE files.

2019-08-27 13:47:12 +09:00

start-build-env.sh

HADOOP-16240. start-build-env.sh can consume all disk space during image creation.

2019-04-10 08:48:11 -07:00

README.txt

For the latest information about Hadoop, please visit our website at:

   http://hadoop.apache.org/

and our wiki, at:

   https://cwiki.apache.org/confluence/display/HADOOP/

Languages

Java 92.9%

C++ 2.8%

C 1.8%

JavaScript 1.1%

Shell 0.5%

Other 0.6%