test usage: help for erasure coding command -usage SubstringComparator Can't understand command '-usage' SubstringComparator Usage: bin/hdfs ec [COMMAND] help: help for erasure coding command -help SubstringComparator [-listPolicies] SubstringComparator [-unsetPolicy -path help: help with extra argument -help arg1 arg2 SubstringComparator You must give exactly one argument to -help. 1 help: setPolicy command -fs NAMENODE -help setPolicy SubstringComparator Set the erasure coding policy for a file/directory. SubstringComparator [-setPolicy -path <path> [-policy <policy>] [-replicate]] help: getPolicy command -fs NAMENODE -help getPolicy SubstringComparator Get the erasure coding policy of a file/directory SubstringComparator [-getPolicy -path <path>] help: listPolicies command -fs NAMENODE -help listPolicies SubstringComparator Get the list of all erasure coding policies SubstringComparator [-listPolicies] help: addPolicies command -fs NAMENODE -help addPolicies SubstringComparator Add a list of user defined erasure coding policies SubstringComparator [-addPolicies -policyFile <file>] help: removePolicy command -fs NAMENODE -help removePolicy SubstringComparator Remove an user defined erasure coding policy SubstringComparator [-removePolicy -policy <policy>] help: enablePolicy command -fs NAMENODE -help enablePolicy SubstringComparator Enable the erasure coding policy SubstringComparator [-enablePolicy -policy <policy>] help: disablePolicy command -fs NAMENODE -help disablePolicy SubstringComparator Disable the erasure coding policy SubstringComparator [-disablePolicy -policy <policy>] setPolicy : set erasure coding policy on a directory to encode files -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator Set RS-6-3-1024k erasure coding policy on /ecdir setPolicy : set a policy twice -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator Set RS-6-3-1024k erasure coding policy on /ecdir setPolicy : set replication policy on a directory -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -mkdir /ecdir/replica -fs NAMENODE -setPolicy -replicate -path /ecdir/replica -fs NAMENODE -touchz /ecdir/replica/file -fs NAMENODE -getPolicy -path /ecdir/replica/file -fs NAMENODE -rm /ecdir/replica/file -fs NAMENODE -rmdir /ecdir/replica -fs NAMENODE -rmdir /ecdir SubstringComparator is unspecified unsetPolicy : unset policy and get -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -unsetPolicy -path /ecdir -fs NAMENODE -getPolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator is unspecified setPolicy : change different policy and get -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -setPolicy -policy RS-3-2-1024k -path /ecdir -fs NAMENODE -getPolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator RS-3-2-1024k setPolicy : set policy on non-empty directory -fs NAMENODE -mkdir /ecdir -fs NAMENODE -touchz /ecdir/file1 -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -rm -R /ecdir SubstringComparator Warning: setting erasure coding policy on a non-empty directory will not automatically convert existing files to RS-6-3-1024k erasure coding policy unsetPolicy : unset inherited EC policy, has no effect -fs NAMENODE -mkdir /ecdir -fs NAMENODE -mkdir /ecdir/child -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -unsetPolicy -path /ecdir/child -fs NAMENODE -touchz /ecdir/child/ecfile -fs NAMENODE -getPolicy -path /ecdir/child/ecfile -fs NAMENODE -rm /ecdir/child/ecfile -fs NAMENODE -rmdir /ecdir/child -fs NAMENODE -rmdir /ecdir SubstringComparator RS-6-3-1024k unsetPolicy : unset on non EC directory -fs NAMENODE -mkdir /ecdir -fs NAMENODE -mkdir /ecdir/child -fs NAMENODE -unsetPolicy -path /ecdir/child -fs NAMENODE -rm /ecdir/child/ecfile -fs NAMENODE -rmdir /ecdir/child -fs NAMENODE -rmdir /ecdir SubstringComparator NoECPolicySetException: No erasure coding policy explicitly set on /ecdir/child SubstringComparator Use '-setPolicy -path <PATH> -replicate' to enforce default replication policy irrespective of EC policy defined on parent. unsetPolicy : unset policy on non-empty directory -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -touchz /ecdir/file1 -fs NAMENODE -unsetPolicy -path /ecdir -fs NAMENODE -rm -R /ecdir SubstringComparator Warning: unsetting erasure coding policy on a non-empty directory will not automatically convert existing files to replicated data getPolicy : get EC policy information at specified path, which doesn't have an EC policy -fs NAMENODE -mkdir /noec -fs NAMENODE -getPolicy -path /noec -fs NAMENODE -rmdir /noec SubstringComparator is unspecified getPolicy : get EC policy information at specified path, which has an EC policy -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -getPolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator RS-6-3-1024k getPolicy : get EC policy information at specified path, which has an EC policy -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-6-3-1024k -path /ecdir -fs NAMENODE -touchz /ecdir/ecfile -fs NAMENODE -getPolicy -path /ecdir/ecfile -fs NAMENODE -rm /ecdir/ecfile -fs NAMENODE -rmdir /ecdir SubstringComparator RS-6-3-1024k listPolicies : get the list of ECPolicies supported -fs NAMENODE -listPolicies SubstringComparator RS-6-3 addPolicies : add a list of ECPolicies -fs NAMENODE -addPolicies -policyFile CLITEST_DATA/test_ec_policies.xml SubstringComparator Add ErasureCodingPolicy XOR-2-1-128k succeed SubstringComparator Add ErasureCodingPolicy RS-6-3-1024k failed listPolicies : get the list of ECPolicies supported -fs NAMENODE -listPolicies SubstringComparator XOR-2-1-128k SubstringComparator State=DISABLED enablePolicy : enable the erasure coding policy -fs NAMENODE -enablePolicy -policy RS-6-3-1024k SubstringComparator Erasure coding policy RS-6-3-1024k is enabled enablePolicy : enable the erasure coding policy twice -fs NAMENODE -enablePolicy -policy RS-6-3-1024k -fs NAMENODE -enablePolicy -policy RS-6-3-1024k SubstringComparator Erasure coding policy RS-6-3-1024k is enabled disablePolicy : disable the erasure coding policy -fs NAMENODE -disablePolicy -policy RS-6-3-1024k -fs NAMENODE -enablePolicy -policy RS-6-3-1024k SubstringComparator Erasure coding policy RS-6-3-1024k is disabled disablePolicy : disable the erasure coding policy twice -fs NAMENODE -disablePolicy -policy RS-6-3-1024k -fs NAMENODE -disablePolicy -policy RS-6-3-1024k -fs NAMENODE -enablePolicy -policy RS-6-3-1024k SubstringComparator Erasure coding policy RS-6-3-1024k is disabled setPolicy : illegal parameters - path option is missing -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -fs NAMENODE -rmdir /ecdir SubstringComparator Please specify the path for setting the EC policy. setPolicy : illegal parameters - path name is missing -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -path -fs NAMENODE -rmdir /ecdir SubstringComparator option -path requires 1 argument setPolicy : illegal parameters - too many arguments case 1 -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -path /ecdir1 -policy RS-3-2-1024k /ecdir2 -fs NAMENODE -rmdir /ecdir SubstringComparator -setPolicy: Too many arguments setPolicy : illegal parameters - too many arguments case 2 -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -path /ecdir1 -policy RS-3-2-1024k -replicate /ecdir2 -fs NAMENODE -rmdir /ecdir SubstringComparator -setPolicy: Too many arguments setPolicy : illegal parameters - invalidpolicy -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy invalidpolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator Policy 'invalidpolicy' does not match any enabled erasure coding policies setPolicy : illegal parameters - RS-10-4-1024k -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -policy RS-10-4-1024k -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator Policy 'RS-10-4-1024k' does not match any enabled erasure coding policies setPolicy : illegal parameters - no such file -fs NAMENODE -setPolicy -path /ecdir -policy RS-3-2-1024k SubstringComparator Path not found: /ecdir setPolicy : illegal parameters - wrong spelling replicate -fs NAMENODE -setPolicy -path /ecdir -replica SubstringComparator -setPolicy: Too many arguments setPolicy : illegal parameters - replicate and policy coexist -fs NAMENODE -setPolicy -path /ecdir -policy RS-3-2-1024k -replicate SubstringComparator -replicate and -policy cannot been used at the same time setPolicy : set erasure coding policy without given a specific policy name -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator Set default erasure coding policy on /ecdir setPolicy : set erasure coding policy without given a specific policy name on a non empty directory -fs NAMENODE -mkdir /ecdir -fs NAMENODE -mkdir /ecdir/ecsubdir -fs NAMENODE -setPolicy -path /ecdir -fs NAMENODE -rm -R /ecdir SubstringComparator Set default erasure coding policy on /ecdir SubstringComparator Warning: setting erasure coding policy on a non-empty directory will not automatically convert existing files to default erasure coding policy getPolicy: get the default policy after setPolicy without given a specific policy name -fs NAMENODE -mkdir /ecdir -fs NAMENODE -setPolicy -path /ecdir -fs NAMENODE -getPolicy -path /ecdir -fs NAMENODE -rmdir /ecdir SubstringComparator RS-6-3-1024k getPolicy : illegal parameters - path is missing -fs NAMENODE -getPolicy SubstringComparator Please specify the path with -path getPolicy : illegal parameters - too many arguments -fs NAMENODE -getPolicy -path /ecdir /ecdir -fs NAMENODE -rm /ecdir SubstringComparator -getPolicy: Too many arguments getPolicy : illegal parameters - no such file -fs NAMENODE -getPolicy -path /ecdir SubstringComparator Path not found: /ecdir listPolicies : illegal parameters - too many parameters -fs NAMENODE -listPolicies /ecdir SubstringComparator -listPolicies: Too many arguments addPolicies : illegal parameters - policyFile is missing -fs NAMENODE -addPolicies /etc SubstringComparator Please specify the path with -policyFile addPolicies : illegal parameters - too many parameters -fs NAMENODE -addPolicies -policyFile /ecdir /ecdir2 SubstringComparator -addPolicies: Too many arguments enablePolicy : illegal parameters - policy is missing -fs NAMENODE -enablePolicy RS-6-3-1024k SubstringComparator Please specify the policy name enablePolicy : illegal parameters - too many parameters -fs NAMENODE -enablePolicy -policy RS-6-3-1024k RS-3-2-1024k SubstringComparator -enablePolicy: Too many arguments disablePolicy : illegal parameters - policy is missing -fs NAMENODE -disablePolicy RS-6-3-1024k SubstringComparator Please specify the policy name disablePolicy : illegal parameters - too many parameters -fs NAMENODE -disablePolicy -policy RS-6-3-1024k RS-3-2-1024k SubstringComparator -disablePolicy: Too many arguments listCodecs : illegal parameters - too many parameters -fs NAMENODE -listCodecs /ecdir SubstringComparator -listCodecs: Too many arguments listCodecs : successful list codecs -fs NAMENODE -listCodecs SubstringComparator Erasure Coding Codecs: Codec [Coder List] count: file using absolute path with option -e to show erasurecoding policy of a directory -fs NAMENODE -mkdir /dir1 -fs NAMENODE -setPolicy -path /dir1 -policy RS-6-3-1024k -fs NAMENODE -touchz /dir1/file1 -fs NAMENODE -touchz /dir1/file2 -fs NAMENODE -count -e -v /dir1 -fs NAMENODE -rmdir /dir1 RegexpComparator ( |\t)*1( |\t)*2( |\t)*0 EC:[A-Za-z0-9-]{1,}( )*/dir1 count: file using absolute path with option -e to show erasurecoding policy of a file and option -v to show head information -fs NAMENODE -touchz /file1 -fs NAMENODE -count -e -v /file1 -fs NAMENODE -rm /file1 RegexpComparator ( |\t)*DIR_COUNT FILE_COUNT CONTENT_SIZE( )*ERASURECODING_POLICY( )*PATHNAME RegexpComparator ( |\t)*0( |\t)*1( |\t)*0 [A-Za-z0-9-]{1,}( )*/file1 ls: file using absolute path and option -e to show erasure coding policy of a directory -fs NAMENODE -mkdir -p /ecdir -fs NAMENODE -setPolicy -path /ecdir -policy RS-6-3-1024k -fs NAMENODE -touchz /ecdir/file1 -fs NAMENODE -touchz /ecdir/file2 -fs NAMENODE -touchz /ecdir/file3 -fs NAMENODE -ls -e /ecdir -fs NAMENODE -rmdir /ecdir RegexpComparator Found [0-9] items RegexpComparator ^-rw-r--r--( )*1( )*USERNAME( )*supergroup( )*[A-Za-z0-9-]{1,}( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/ecdir/file1 RegexpComparator ^-rw-r--r--( )*1( )*USERNAME( )*supergroup( )*[A-Za-z0-9-]{1,}( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/ecdir/file2 RegexpComparator ^-rw-r--r--( )*1( )*USERNAME( )*supergroup( )*[A-Za-z0-9-]{1,}( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/ecdir/file3 ls: file with disabled EC Policy -fs NAMENODE -mkdir -p /ecdir -fs NAMENODE -setPolicy -path /ecdir -policy XOR-2-1-1024k -fs NAMENODE -touchz /ecdir/file1 -fs NAMENODE -disablePolicy -policy XOR-2-1-1024k -fs NAMENODE -ls -e /ecdir -fs NAMENODE -rmdir /ecdir -fs NAMENODE -enablePolicy -policy XOR-2-1-1024k RegexpComparator ^-rw-r--r--( )*1( )*USERNAME( )*supergroup( )*[A-Za-z0-9-]{1,}( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/ecdir/file1 ls: file with replication Policy -fs NAMENODE -mkdir -p /ecdir -fs NAMENODE -setPolicy -path /ecdir -replicate -fs NAMENODE -touchz /ecdir/file1 -fs NAMENODE -ls -e / -fs NAMENODE -rmdir /ecdir RegexpComparator ^drwxr-xr-x( )*-( )*USERNAME( )*supergroup( )*[A-Za-z0-9-]{1,}( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*/ecdir