Can pyhdfs make a 'soft' delete?

170 views Asked by At

I am using

from pyhdfs import HdfsClient

fs = HdfsClient(hosts=..., user_name='hdfs', ..)
fs.delete(path_table, recursive=True)

However, after I deleted the directory, I could not find it in the trash directory located in /user/hdfs/.Trash/Current/.

Does it look like the pyhdfs deletes the files with '-skipTrash'? How to make it softly delete the target? I could not find method like '-mv' to implement this request manually, either.

This is the description from the official document: enter image description here

1

There are 1 answers

0
AKX On BEST ANSWER

No skipTrash-esque option (or an alternative to not skip the trash) is documented for the WebHDFS delete command.

The hdfs dfs rm command primarily attempts to use the Trash API (unless told not to), and if trashing fails, it deletes the file using what I'd assume is the same underlying operation the WebHDFS API uses.

Trashing seems to be implemented ((1), (2)) as a create-directories-and-rename command.

It's likely you would need to implement that sequence by hand when using WebHDFS (and by extension pyhdfs).