Hadoop fs du

The hadoop fs -du command is used to display the disk usage of files and directories in Hadoop HDFS (Hadoop Distributed File System). It shows the amount of storage space occupied by the specified paths in HDFS. Here’s how you can use the hadoop fs -du command:

Syntax:

• -s (optional): If you include the -s option, the command will display a summary of the total disk usage of the specified path, rather than listing the usage for individual subdirectories and files within that path.

• -h (optional): If you include the -h option, the command will display the sizes in a human-readable format (e.g., KB, MB, GB) rather than in bytes.

• <path>: This is the HDFS path for which you want to check the disk usage.

Examples:

1. To display the disk usage of a specific directory in HDFS, you can use the following command:

   bashCopy code

   hadoop fs -du /user/yourusername/some_directory

   This will list the sizes of all subdirectories and files within the /user/yourusername/some_directory in HDFS.

2. To get a summary of the total disk usage for a directory and its subdirectories, you can use the -s option:

   bashCopy code

   hadoop fs -du -s /user/yourusername/some_directory

   This will provide a single total size for the specified directory and all its contents.

3. To display the disk usage in a human-readable format, you can combine the -h option with the previous commands:

   bashCopy code

   hadoop fs -du -sh /user/yourusername/some_directory