big-data/hadoop
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
27,462 Commits 2 Branches 2 Tags 264 MiB
66baf1eb51
Commit Graph

281 Commits

Author SHA1 Message Date
James
6dd47cae86 libhdfs++: Get rid of lock in RpcConnectionImpl destructor. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James
7ebecaeede HDFS-10898: libhdfs++: Make log levels consistent. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James
2a42eeb66f HDFS-10931: libhdfs++: Fix object lifecycle issues in the BlockReader (see follow up work in jira). Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
Bob Hansen
69cb05d292 HDFS-10595: libhdfs++: deconflate client name and client id 2018-03-22 17:19:47 -04:00
Bob Hansen
d22e4b2eb7 HDFS-10450b: libhdfs++: Add Cyrus SASL support (engines). Contributed by Don Davis. 2018-03-22 17:19:47 -04:00
Bob Hansen
549a5dbce9 HDFS-10450: libhdfs++: Add Cyrus SASL support. Contributed by Don Davis. 2018-03-22 17:19:47 -04:00
James
9d1c902159 HDFS-10705: libhdfs++: FileSystem should have a convenience no-args ctor. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James
4f6cb5d1a1 HDFS-10754: libhdfs++: Create tools directory and implement hdfs_cat, hdfs_chgrp, hdfs_chown, hdfs_chmod and hdfs_find. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
05ddb31081 HDFS-10761: libhdfs++: Fix broken logic in HA retry policy. Contributed by James Clampffer 2018-03-22 17:19:47 -04:00
James
b9cf0e932d HDFS-10746: libhdfs++: synchronize access to working_directory and bytes_read_. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
cbb3e9fe58 HDFS-10740: libhdfs++: Implement recursive directory generator. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
2a8edd4e52 HDFS-9271: libhdfs++: Implement basic NN operations. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
b1ed72e098 HDFS-10739: libhdfs++: In RpcEngine replace vector with deque for pending requests. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
649aff11fe HDFS-10672: libhdfs++: reorder directories in src/main/libhdfspp/examples, and add C++ version of cat tool. Contributed by Anatoli Shein. 2018-03-22 17:19:47 -04:00
James
4cb0dad5e5 HDFS-10441: libhdfs++: HA namenode support. Contributed by James Clampffer. 2018-03-22 17:19:47 -04:00
James
d75c835696 Revert HDFS-10543 patch due to correctness issues it caused. Done by James Clampffer 2018-03-22 17:19:47 -04:00
James
a586ccbcfa HDFS-9890: libhdfs++: Add test suite to simulate network issues. Contributed by Xiaowei Zhu. 2018-03-22 17:19:47 -04:00
James
a23e6b422b HDFS-10578. libhdfs++: Silence compile warnings from URI parser. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
a0c7aee3b6 HDFS-10543: libhdfs++: hdfsRead stops at block boundary. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
Bob Hansen
d4c3cfbf47 HDFS-10524: libhdfs++: Implement chmod and chown. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
88c5768f99 HDFS-10515: libhdfs++: Implement mkdirs, rmdir, rename, and remove 2018-03-22 17:19:46 -04:00
Bob Hansen
193314dc34 HDFS-10511: libhdfs++: make error returning mechanism consistent across all hdfs operations. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
2616fe2025 HDFS-10526: libhdfs++: Add connect timeouts to async_connect calls. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
de6fce7817 HDFS-10527: libhdfs++: hdfsGetBlockLocations doesn't null terminate ip address strings. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
Bob Hansen
9ac66ccaa3 HDFS-10494: libhdfs++: Implement snapshot operations and GetFsStats. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
James
d64bbf983e Revert HDFS-9890. Needs some more work to reflect the current status of HDFS-8707. Done by James Clampffer 2018-03-22 17:19:46 -04:00
James
11b68ecebe HDFS-9890. libhdfs++: Add test suite to simulate network issues. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
Bob Hansen
50aa4ea776 HDFS-10454: libhdfspp: Move NameNodeOp to a separate file. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
0496bc5464 HDFS-10464: libhdfs++: Implement GetPathInfo and ListDirectory. Contributed by Anatoli Shein. 2018-03-22 17:19:46 -04:00
Bob Hansen
18f4d2f42e HDFS-10465: libhdfs++: Implement GetBlockLocations. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
f1f0b8f0f8 HDFS-10366: libhdfs++: Add SASL authentication. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
93382381f6 HDFS-10188. libhdfs++: Implement debug allocators. Contributed by Xiaowei Zhu 2018-03-22 17:19:46 -04:00
James
b9eeecc477 HDFS-10332. hdfs-native-client fails to build with CMake 2.8.11 or earlier. Contributed by Tibor Kiss 2018-03-22 17:19:46 -04:00
James
cc51f2ce58 HDFS-10310. hdfsConnect hangs when given bad host or port. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
e542db66ee HDFS-10311: DatanodeConnection::Cancel should not delete the underlying socket. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
1cbfd6f962 libhdfs++: File length doesn't always count the last block if it's being written to. Contributed by Xiaowei Zhu. 2018-03-22 17:19:46 -04:00
James
60c3437267 HDFS-10247: libhdfs++: Datanode protocol version mismatch fix. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
ed77d8d5df HDFS-10231: libhdfs++: Fix race conditions in RPC layer. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
015d93a715 HDFS-10222. libhdfs++: Shutdown sockets to avoid 'Connection reset by peer'. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
427edae365 HDFS-9616. libhdfs++: Add runtime hooks to allow a client application to add low level monitoring and tests. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
0f1a278dd5 HDFS-9118: libhdfs++ Add logging system. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
8f4a66ab8f HDFS-9556: libhdfs++: pull Options from default configs by default. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
Bob Hansen
f25bff50bf HDFS-9932: added uriparser2 library to HDFS-8707. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
Bob Hansen
69ee2e6a7e HDFS-9792: libhdfs++: EACCES not setting errno correctly. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
6a96f978eb HDFS-9699: libhdfs++: Add appropriate catch blocks for asio operations that throw. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
1600e6bba8 HDFS-9791: libhdfs+_+: ConfigurationLoader throws parse_exception on invalid input. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
2cd7ab5e94 HDFS-9753. libhdfs++: Client fails to pass TokenProto from LocatedBlockProto to server when reading a block. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
Bob Hansen
22ae319360 HDFS-9737: libhdfs++: Create examples of consuming libhdfs++ (pt 2). Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
f591b95e08 HDFS-9749: libhdfs++: RPC engine will attempt to close an asio socket before it's been opened. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
Bob Hansen
288a9a9ee7 HDFS-9741: libhdfs++: GetLastError not returning meaningful messages after some failures. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
d807773752 HDFS-9738: libhdfs++: Implement simple authentication. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
71b4f2ea25 HDFS-9737: libhdfs++: Create examples of consuming libhdfs++. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
Bob Hansen
7e946c7e30 HDFS-9712. libhdfs++: Reimplement Status object as a normal struct 2018-03-22 17:19:46 -04:00
Bob Hansen
ad23cf1490 HDFS-9679: Fix inconsistencies with libhdfs C API. Contributed by James Clampffer 2018-03-22 17:19:46 -04:00
James
166b3d49df HDFS-9643. libhdfs++: Support async cancellation of read operations. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
5cd2a1766c HDFS-9687. libhdfs++: potential segfault after teardown. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
1017ccabeb HDFS-9628. libhdfs++: implement builder apis from C bindings. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
dc335474a6 HDFS-9636. libhdfs++: for consistency, include files should be in hdfspp. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
11671c2bf6 HDFS-9627. libhdfs++: Add mechanism to retrieve human readable error messages through the C API. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
e376c022f4 HDFS-9609. libhdfs++: Allow seek to EOF. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
7d8452040d HDFS-9325. libhdfs++ Allow the location of hadoop source tree resources to be passed to CMake during a build. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
5dc2da1e6f HDFS-9487. libhdfs++ Enable builds with no compiler optimizations. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
e18db92398 HDFS-9523. libhdfs++: failure to connect to ipv6 host causes CI unit tests to fail. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
31d28e3105 HDFS-9524. libhdfs++ deadlocks in Filesystem::New if NN connection fails. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
dba377bfcc HDFS-9448. Enable valgrind for libhdfspp unit tests. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
6c80d02a03 HDFS-9538. libhdfs: load configuration from files. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
567c3ed44f HDFS-9537. libhdfs++ implement HDFSConfiguration class. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
6c33f89aea HDFS-9417. Clean up the RAT warnings in the HDFS-8707 branch. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
7cd98323ec HDFS-9489. Enable CI infrastructure to use libhdfs++ hdfsRead. Contributed by Stephen Walkauskas. 2018-03-22 17:19:46 -04:00
James
d6a04cdbdd HDFS-9460. Suppress warnings from third-party libraries. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
c59bd00a56 HDFS-9486. Fix valgrind failures when using more than 1 io_service worker thread. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
58f2c7183e HDFS-9228. libhdfs++ should respect NN retry configuration settings. Contributed by Bob Hansen 2018-03-22 17:19:46 -04:00
James
eb70a64362 HDFS-9504. Initialize BadNodeTracker in FileSystemImpl constructor. Contributed by James Clampffer. 2018-03-22 17:19:46 -04:00
James
8dac1094d8 HDFS-9497. move lib/proto/cpp_helpers to third-party since it won't have an ASF license. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
d7ecf396c9 HDFS-9144. Refactoring libhdfs++ into stateful/ephemeral objects. Contributed by Bob Hansen. 2018-03-22 17:19:46 -04:00
James
5ba8e8ab7b Revert HDFS-9448. 2018-03-22 17:19:46 -04:00
James
9a0a34bbfa HDFS-9448. Enable valgrind for libhdfspp unit tests. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
James
584c2a204d HDFS-9452. libhdfs++ Fix memory stomp in OpenFileForRead. Contributed by James Clampffer 2018-03-22 17:19:45 -04:00
James
87362b1c17 HDFS-9117. Config file reader / options classes for libhdfs++. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
James
a38703fdfc HDFS-9359. Test libhdfs++ with existing libhdfs tests. Contributed by Stephen Walkauskas. 2018-03-22 17:19:45 -04:00
James
64f0e78490 HDFS-9368. Implement reads with implicit offset state in libhdfs++. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
6f44d92071 HDFS-9103. Retry reads on DN failure. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
7dd29e548c HDFS-9408. Build both static and dynamic libraries for libhdfspp. Contributed by Stephen Walkauskas. 2018-03-22 17:19:45 -04:00
Haohui Mai
af9bc25ac5 HDFS-9419. Import the optional library into libhdfs++. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
Haohui Mai
a0b8f56317 HDFS-9328. Formalize coding standards for libhdfs++. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
a59714dcdb HDFS-9320. libhdfspp should use sizeof(int32_t) instead of sizeof(int) when parsing data. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
dbd1d0c75b HDFS-8766. Implement a libhdfs(3) compatible API. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
422a6e4ed5 HDFS-9288. Import RapidXML 1.13 for libhdfspp. Contributed by Bob Hansen. 2018-03-22 17:19:45 -04:00
Haohui Mai
a14d1f741f HDFS-9265. InputStreamImpl should hold a shared_ptr of the BlockReader. Contributed by James Clampffer. 2018-03-22 17:19:45 -04:00
Haohui Mai
08794423e8 HDFS-9207. Move the implementation to the hdfs-native-client module. Contributed by Haohui Mai. 2018-03-22 17:19:45 -04:00
Chris Douglas
3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Wangda Tan
60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
John Zhuge
b8c8b5bc27 HDFS-12714. Hadoop 3 missing fix for HDFS-5169. Contributed by Joe McDonnell. 2017-10-31 22:44:16 -07:00
John Zhuge
2856eb207b HDFS-12494. libhdfs SIGSEGV in setTLSExceptionStrings. Contributed by John Zhuge. 2017-10-06 23:29:28 -07:00
Andrew Wang
0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Allen Wittenauer
71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms 
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2017-08-30 21:47:24 -07:00
Andrew Wang
af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
John Zhuge
164c222c4a HDFS-11851. getGlobalJNIEnv() may deadlock if exception is thrown. Contributed by Sailesh Mukil. 2017-06-08 14:15:09 -07:00
Andrew Wang
16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Vinayakumar B
b0f54ea035 HDFS-11710. hadoop-hdfs-native-client build fails in trunk in Windows after HDFS-11529 (Contributed by Sailesh Mukil) 2017-05-02 10:32:38 +05:30
John Zhuge
343948ca79 HDFS-11724. libhdfs compilation is broken on OS X. Contributed by John Zhuge. 2017-05-01 13:37:55 -07:00
John Zhuge
fda86ef2a3 HDFS-11529. Add libHDFS API to return last exception. Contributed by Sailesh Mukil. 2017-04-22 02:56:48 -07:00
John Zhuge
8c81a16a1f HDFS-11531. Expose hedged read metrics via libHDFS API. Contributed by Sailesh Mukil. 2017-04-18 21:11:07 -07:00
Andrew Wang
5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Wei-Chiu Chuang
c51bfd29cd HDFS-11181. Fuse wrapper has a typo. Contributed by Wei-Chiu Chuang. 2016-12-05 08:44:40 -08:00
Akira Ajisaka
209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Andrew Wang
dacd3ec66b HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore. 2016-10-11 15:07:14 -07:00
Akira Ajisaka
2d1bf53c7e HDFS-10844. test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing. 2016-09-09 00:49:22 +09:00
Andrew Wang
da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Akira Ajisaka
ff07b10803 HDFS-10561. test_native_mini_dfs fails by NoClassDefFoundError. (aajisaka) 2016-06-23 15:48:18 +09:00
Andrew Wang
3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang
ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang
6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Andrew Wang
3ff0510ffd Fix hadoop-hdfs-native-client compilation on Windows. Contributed by Brahma Reddy Battula. 2016-05-02 18:15:51 -07:00
Allen Wittenauer
7b1c37a13a HADOOP-12892. fix/rewrite create-release (aw) 2016-04-27 08:38:22 -07:00
Allen Wittenauer
0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Colin Patrick Mccabe
0d19a0ce98 HDFS-10193. fuse_dfs segfaults if uid cannot be resolved to a username (John Thiltges via cmccabe) 2016-03-23 11:19:18 -07:00
Zhe Zhang
cf8af7bb45 HDFS-9541. Add hdfsStreamBuilder API to libhdfs to support defaultBlockSizes greater than 2 GB. Contributed by Colin Patrick McCabe. 
Change-Id: Ifce1b9be534dc8f7e9d2634cd60e423921b9810f
 2016-01-26 11:24:57 -08:00
Colin Patrick Mccabe
b1ed28fa77 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 2016-01-14 11:02:34 -08:00
Haohui Mai
c213ee0859 HDFS-9047. Retire libwebhdfs. Contributed by Haohui Mai. 2016-01-06 16:16:31 -08:00
Zhe Zhang
1cc7e61431 HDFS-9269. Update the documentation and wrapper for fuse-dfs. Contributed by Wei-Chiu Chuang. 
Change-Id: Ia9ec512de2464bf94725cc7c15c378c59d0f04c4
 2015-12-01 10:24:49 -08:00
Haohui Mai
95d5227c75 HDFS-9459. hadoop-hdfs-native-client fails test build on Windows after transition to ctest. Contributed by Chris Nauroth. 2015-11-25 10:32:48 -08:00
Haohui Mai
a06e6b81e0 HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. Contributed by Haohui Mai. 2015-11-10 14:53:05 -08:00
Colin Patrick Mccabe
faeb6a3f89 HDFS-9307. fuseConnect should be private to fuse_connect.c (Mingliang Liu via Colin P. McCabe) 2015-10-27 11:41:05 -07:00
Colin Patrick Mccabe
2f1eb2bceb HDFS-9268. fuse_dfs chown crashes when uid is passed as -1 (cmccabe) 2015-10-26 13:33:22 -07:00
Haohui Mai
79b8d60d08 HDFS-9253. Refactor tests of libhdfs into a directory. Contributed by Haohui Mai. 2015-10-16 11:20:17 -07:00
Haohui Mai
eb50c4f7a2 HDFS-9235. hdfs-native-client build getting errors when built with cmake 2.6. Contributed by Eric Payne. 2015-10-13 13:44:38 -07:00
Haohui Mai
c60a16fcea Addendum patch for HDFS-9215. 2015-10-12 15:52:10 -07:00
Haohui Mai
18950c0a55 HDFS-9222. Add hadoop-hdfs-client as a dependency of hadoop-hdfs-native-client. Contributed by Mingliang Liu. 2015-10-09 22:31:16 -07:00
Haohui Mai
d0c1e5fec4 HDFS-9215. Suppress the RAT warnings in hdfs-native-client module. Contributed by Haohui Mai. 2015-10-09 21:12:44 -07:00
Haohui Mai
3112f26368 HDFS-9170. Move libhdfs / fuse-dfs / libwebhdfs to hdfs-client. Contributed by Haohui Mai. 2015-10-07 00:06:33 -07:00