Writing with an OutputStream to a DocumentFile: data seem to be written but file ends up empty

Question

The application KDE Connect allows remotely browsing an Android device from a desktop computer through SFTP. Since Android 4.4, developers don't have write permission to SD cards directly through the filesystem anymore. So I am trying to port the SFTP module using the Storage Access Framework (DocumentFile, etc.)

I am taking the permission with an Intent.ACTION_OPEN_DOCUMENT_TREE and FLAG_GRANT_WRITE_URI_PERMISSION and passing the context to my classes.

I am able to create new empty files, rename files and delete files on the SD card inside my class so I believe I am getting the necessary permissions. However, transferring a file results in an empty file (0 bytes) being created. I can see the transfer taking a certain time and a progress bar on the desktop side, so it doesn't just abort.

Here is the relevant part of the SftpSubsystem class from the Apache SSHD library (see doc here) with my own comments to explain what's going on:

public class SftpSubsystem implements Command, Runnable, SessionAware, FileSystemAware {
    // This method receives a buffer from an InputStream and processes it
    // according to its type. In this situation, it would also contain
    // a block of the file being transferred (4096 bytes)
    protected void process(Buffer buffer) {
        int type = buffer.getByte();

        switch (type) {
            case WRITE:
                FileHandle fh = getHandleFromString(buffer.getString());
                long offset = buffer.getLong();
                byte[] data = buffer.getBytes();

                fh.write(data, offset);
                break;
            // other cases
        }   
    }

    // This class is a handle to a file (duh) with
    // an OutputStream to write and InputStream to read
    protected static class FileHandle {
        SshFile file;
        OutputStream output;
        long outputPos;
        InputStream input;
        long inputPos;

        // Method called inside process()
        public void write(byte[] data, long offset) throws IOException {
            if (output != null && offset != outputPos) {
                IoUtils.closeQuietly(output);
                output = null;
            }
            if (output == null) {
                // This is called once at the start of the transfer.
                // This is what I think I need to rewrite to make
                // it work with DocumentFile objects.
                output = file.createOutputStream(offset);
            }
            output.write(data);
            outputPos += data.length;
        }
    }
}

The original implementation of createOutputStream() that I want to rewrite because RandomAccessFile doesn't work with DocumentFile:

public class NativeSshFile implements SshFile {
    private File file;

    public OutputStream createOutputStream(final long offset)
            throws IOException {

        // permission check
        if (!isWritable()) {
            throw new IOException("No write permission : " + file.getName());
        }

        // move to the appropriate offset and create output stream
        final RandomAccessFile raf = new RandomAccessFile(file, "rw");
        try {
            raf.setLength(offset);
            raf.seek(offset);

            // The IBM jre needs to have both the stream and the random access file
            // objects closed to actually close the file
            return new FileOutputStream(raf.getFD()) {
                public void close() throws IOException {
                    super.close();
                    raf.close();
                }
            };
        } catch (IOException e) {
            raf.close();
            throw e;
        }
    }
}

One of the ways I tried to implement it:

class SimpleSftpServer {
    static class AndroidSshFile extends NativeSshFile {

        // This is the DocumentFile that is stored after
        // create() created the empty file
        private DocumentFile docFile;

        public OutputStream createOutputStream(final long offset) throws IOException {
            // permission check
            if (!isWritable()) {
                throw new IOException("No write permission : " + docFile.getName());
            }

            ParcelFileDescriptor pfd = context.getContentResolver().openFileDescriptor(docFile.getUri(), "rw");
            FileDescriptor fd = pfd.getFileDescriptor();
            try {
                android.system.Os.lseek(fd, offset, OsConstants.SEEK_SET);
            } catch (ErrnoException e) {
                Log.e("SimpleSftpServer", "" + e);
                return null;
            }

            return new FileOutputstream(fd, offset);
        }
    }
}

I also tried a simple (the offset is ignored but it's just a test):

    public OutputStream createOutputStream(final long offset) throws IOException {
        // permission check
        if (!isWritable()) {
            throw new IOException("No write permission : " + docFile.getName());
        }

        return context.getContentResolver().openOutputStream(docFile.getUri());
    }

I also tried with a FileChannel and to flush and sync the FileOutputStream.

Any idea why I end up with an empty file?

EDIT: here is a small example of a test I did to just write a new file from an existing file. It works, but this is not what I actually want to do (see code above) but I thought I'd provide an example to show that I understand the basics of how to write to an OutputStream.

private void createDocumentFileFromFile() {
        File fileToRead = new File("/storage/0123-4567/lady.m4a");
        File fileToWrite = new File("/storage/0123-4567/lady2.m4a");
        File dir = fileToWrite.getParentFile();
        DocumentFile docDir = DocumentFile.fromTreeUri(context, SimpleSftpServer.externalStorageUri);

        try {
            DocumentFile createdFile = docDir.createFile(null, fileToWrite.getName());
            Uri uriToRead = Uri.fromFile(fileToRead);
            InputStream in = context.getContentResolver().openInputStream(uriToRead);
            OutputStream out = context.getContentResolver().openOutputStream(createdFile.getUri());
            try {
                int nbOfBytes = 0;
                final int BLOCKSIZE = 4096;
                byte[] bytesRead = new byte[BLOCKSIZE];
                while (true) {
                    nbOfBytes = in.read(bytesRead);
                    if (nbOfBytes == -1) {
                        break;
                    }
                    out.write(bytesRead, 0, nbOfBytes);
                }
            } finally {
                in.close();
                out.close();
            }
        } catch (IOException e) {

        }
    }

Answer 1

"When using ACTION_OPEN_DOCUMENT_TREE, your app gains access only to the files in the directory that the user selects. You don't have access to other apps' files that reside outside this user-selected directory.

This user-controlled access allows users to choose exactly what content they're comfortable sharing with your app."

This means, you can only read/write/delete the content/meta data of already existing files or in sub directories in the selected directory, the scope that the user accept to be "comfortable" with.

Actually the user granted permmision to a list of Uri's in this folder for ea file/sub directory there is seperate uri permmision.

Now for example if I will try to create new file in the selected Uri using DocumentFile Ill success but if i will try to outputatream new data to this file I will fail because the user did not grant permision to write to this newly created file.

He only granted to write in the directory path level, means create new file here.

So same happens when you try to move/transfer file to other path that does not have permission from the user.

Path can be folder or file and for ea new path the user needs to grant new access.

move file = new path write to just created file = new path