BREAKDOWN OF HDFS-2832 ENABLE SUPPORT FOR HETEROGENEOUS STORAGES IN HDFS

IMPROVEMENTS:

    HDFS-4985. Add storage type to the protocol and expose it in block report
    and block locations. (Arpit Agarwal)

    HDFS-5115. Make StorageID a UUID. (Arpit Agarwal)

    HDFS-5000. DataNode configuration should allow specifying storage type.
    (Arpit Agarwal)

    HDFS-4987. Namenode changes to track multiple storages per datanode.
    (szetszwo)

    HDFS-5154. Fix TestBlockManager and TestDatanodeDescriptor after HDFS-4987.
    (Junping Du via szetszwo)

    HDFS-5009. Include storage information in the LocatedBlock.  (szetszwo)

    HDFS-5134. Move blockContentsStale, heartbeatedSinceFailover and
    firstBlockReport from DatanodeDescriptor to DatanodeStorageInfo; and
    fix a synchronization problem in DatanodeStorageInfo.  (szetszwo)

    HDFS-5157. Add StorageType to FsVolume.  (Junping Du via szetszwo)

    HDFS-4990. Change BlockPlacementPolicy to choose storages instead of
    datanodes.  (szetszwo)

    HDFS-5232. Protocol changes to transmit StorageUuid. (Arpit Agarwal)

    HDFS-5233. Use Datanode UUID to identify Datanodes. (Arpit Agarwal)

    HDFS-5222. Move block schedule information from DatanodeDescriptor to
    DatanodeStorageInfo.  (szetszwo)

    HDFS-4988. Datanode must support all the volumes as individual storages.
    (Arpit Agarwal)

    HDFS-5377. Heartbeats from Datandode should include one storage report
    per storage directory. (Arpit Agarwal)

    HDFS-5398. NameNode changes to process storage reports per storage
    directory. (Arpit Agarwal)

    HDFS-5390. Send one incremental block report per storage directory.
    (Arpit Agarwal)

    HDFS-5401. Fix NPE in Directory Scanner. (Arpit Agarwal)

    HDFS-5417. Fix storage IDs in PBHelper and UpgradeUtilities.  (szetszwo)

    HDFS-5214. Fix NPEs in BlockManager and DirectoryScanner. (Arpit Agarwal)

    HDFS-5435. File append fails to initialize storageIDs. (Junping Du via
    Arpit Agarwal)

    HDFS-5437. Fix TestBlockReport and TestBPOfferService failures. (Arpit
    Agarwal)

    HDFS-5447. Fix TestJspHelper. (Arpit Agarwal)

    HDFS-5452. Fix TestReplicationPolicy and TestBlocksScheduledCounter.

    HDFS-5448. Datanode should generate its ID on first registration. (Arpit
    Agarwal)

    HDFS-5448. Fix break caused by previous checkin for HDFS-5448. (Arpit
    Agarwal)

    HDFS-5455. NN should update storageMap on first heartbeat. (Arpit Agarwal)

    HDFS-5457. Fix TestDatanodeRegistration, TestFsck and TestAddBlockRetry.
    (Contributed by szetszwo)

    HDFS-5466. Update storage IDs when the pipeline is updated. (Contributed
    by szetszwo)

    HDFS-5439. Fix TestPendingReplication. (Contributed by Junping Du, Arpit
    Agarwal)

    HDFS-5470. Add back trunk's reportDiff algorithm to the branch.
    (Contributed by szetszwo)

    HDFS-5472. Fix TestDatanodeManager, TestSafeMode and
    TestNNThroughputBenchmark (Contributed by szetszwo)

    HDFS-5475. NN incorrectly tracks more than one replica per DN. (Arpit
    Agarwal)

    HDFS-5481. Fix TestDataNodeVolumeFailure in branch HDFS-2832. (Contributed
    by Junping Du)

    HDFS-5480. Update Balancer for HDFS-2832. (Contributed by szetszwo)

    HDFS-5486. Fix TestNameNodeMetrics for HDFS-2832. (Arpit Agarwal)

    HDFS-5491. Update editsStored for HDFS-2832. (Arpit Agarwal)

    HDFS-5494. Fix findbugs warnings for HDFS-2832. (Arpit Agarwal)

    HDFS-5508. Fix compilation error after merge. (Contributed by szetszwo)

    HDFS-5501. Fix pendingReceivedRequests tracking in BPServiceActor. (Arpit
    Agarwal)

    HDFS-5510. Fix a findbug warning in DataStorage.java on HDFS-2832 branch.
    (Junping Du via Arpit Agarwal)
 
    HDFS-5515. Fix TestDFSStartupVersions for HDFS-2832. (Arpit Agarwal)

    HDFS-5527. Fix TestUnderReplicatedBlocks on branch HDFS-2832. (Arpit
    Agarwal)

    HDFS-5547. Fix build break after merge from trunk to HDFS-2832. (Arpit
    Agarwal)

    HDFS-5542. Fix TODO and clean up the code in HDFS-2832. (Contributed by
    szetszwo)

    HDFS-5559. Fix TestDatanodeConfig in HDFS-2832. (Contributed by szetszwo)