"dvc add -external S3://mybucket/data.csv" is failing with access error even after giving correct remote cache configurations

Question

I'm using dvc and connecting to remote S3 for data track and also setting remote dvc cache in same remote S3. Following is configure file,

[core]
    remote = s3remote
[cache]
    s3 = s3cache
[‘remote “s3remote”’]
    url = S3://dvc-example
    endpointurl = http://localhost:9000/
    access_key_id = user
    secret_access_key = password
    use_ssl = false
[‘remote “s3cache”’]
    url = s3://dvc-example/cache
    endpointurl = http://localhost:9000/
    access_key_id = user
    secret_access_key = password
    use_ssl = false

I'm able to push and pull from remote repository to local using s3remote.

But when I try to add external data by configuring cache(s3cache), am getting error.

Both s3cache, s3remote has same credentials, then why is it failing when I add external data in dvc?

Answer 1

We need to use the special remote:// URL format to add --external data using existing remote configurations:

dvc add --external remote://s3remote/wine-quality.csv

wine-quality.csv is in location s3://dvc-example/wine-quality.csv, here remote://s3remote is replaced by url string in config file.

I got answer after posting question in DVC forum. Link to answer