NuGet/MyGet debug throws source code not found error

Question

We are trying to adapt Nuget package way for our project. All is well till we hit debugging problem. Here is the situation:

1. Created New project and created nuget package with symbols
2. Pushed both nuget package and symbols to MyGet Server. I could see source files are listed in myget server.
3. Set Options -> Debugging -> General - Uncheck Enable just my code
4. Set Options -> Debugging -> Symbols to MyGet Server url
5. Also set Cache symbols in local folder
6. Set Options -> NuGet Package Manager -> Package manager sources to MyGet Server

In my Test project, I could see my Nuget package and debug working fine as source code is in same machine

On Client machine (Another machine), created test project and try to debug the package and it is throwing cs file not found error.

When I check the Cache symbols local folder I see only .pdb file downloaded, but no files from src folder are downloaded - where all the cs files are available.

Am I missing anything in this? Why my debug is failing why Visual Studio 2017 not downloading src files from myget server?