YouTube C# API V3, how do you resume an interrupted upload?

Question

I can't work out how to resume an interrupted upload in V3 of the C# YouTube API.

My existing code uses V1 and works fine but I'm switching to V3.

If I call UploadAsync() without changing anything, it starts from the beginning. Using Fiddler, I can see the protocol given here is not followed and the upload restarts.

I've tried setting the position within the stream as per V1 but there is no ResumeAsync() method available.

The Python example uses NextChunk but the SendNextChunk method is protected and not available in C#.

In the code below, both UploadVideo() and Resume() work fine if I leave them to completion but the entire video is uploaded instead of just the remaining parts.

How do I resume an interrupted upload using google.apis.youtube.v3?

Here is the C# code I have tried so far.

private ResumableUpload<Video> UploadVideo(
    YouTubeService youTubeService, Video video, Stream stream, UserCredential userCredentials)
{
    var resumableUpload = youTubeService.Videos.Insert(video, 
        "snippet,status,contentDetails", stream, "video/*");
    resumableUpload.OauthToken = userCredentials.Token.AccessToken;
    resumableUpload.ChunkSize = 256 * 1024;
    resumableUpload.ProgressChanged += resumableUpload_ProgressChanged;
    resumableUpload.ResponseReceived += resumableUpload_ResponseReceived;                   
    resumableUpload.UploadAsync();
    return resumableUpload;
}

private void Resume(ResumableUpload<Video> resumableUpload)
{   
    //I tried seeking like V1 but it doesn't work
    //if (resumableUpload.ContentStream.CanSeek)
    //  resumableUpload.ContentStream.Seek(resumableUpload.ContentStream.Position, SeekOrigin.Begin);

    resumableUpload.UploadAsync(); // <----This restarts the upload                             
}

void resumableUpload_ResponseReceived(Video obj)
{                   
    Debug.WriteLine("Video status: {0}", obj.Status.UploadStatus);                      
}

void resumableUpload_ProgressChanged(IUploadProgress obj)
{
    Debug.WriteLine("Position: {0}", (resumableUploadTest == null) ? 0 : resumableUploadTest.ContentStream.Position);   
    Debug.WriteLine("Status: {0}", obj.Status);
    Debug.WriteLine("Bytes sent: {0}", obj.BytesSent);
}

private void button2_Click(object sender, EventArgs e)
{
    Resume(resumableUploadTest);
}

Any solution/suggestion/demo or a link to the "google.apis.youtube.v3" source code will be very helpful.

Thanks in Advance !

EDIT: New information

I'm still working on this and I believe the API simply isn't finished. Either that or I'm missing something simple.

I still can't find the "google.apis.youtube.v3" source code so I downloaded the latest "google-api-dotnet-client" source code. This contains the ResumableUpload class used by the YouTube API.

I managed to successfully continue an upload by skipping the first four lines of code in the UploadAsync() method. I created a new method called ResumeAsync(), a copy of UploadAsync() with the first four lines of initialization code removed. Everything worked and the upload resumed from where it was and completed.

I'd rather not be changing code in the API so if anyone knows how I should be using this, let me know.

I'll keep plugging away and see if I can work it out.

This is the original UploadAsync() method and my ResumeAsync() hack.

public async Task<IUploadProgress> UploadAsync(CancellationToken cancellationToken)
{
    try
    {
        BytesServerReceived = 0;
        UpdateProgress(new ResumableUploadProgress(UploadStatus.Starting, 0));
        // Check if the stream length is known.
        StreamLength = ContentStream.CanSeek ? ContentStream.Length : UnknownSize;
        UploadUri = await InitializeUpload(cancellationToken).ConfigureAwait(false);

        Logger.Debug("MediaUpload[{0}] - Start uploading...", UploadUri);

        using (var callback = new ServerErrorCallback(this))
        {
            while (!await SendNextChunkAsync(ContentStream, cancellationToken).ConfigureAwait(false))
            {
                UpdateProgress(new ResumableUploadProgress(UploadStatus.Uploading, BytesServerReceived));
            }
            UpdateProgress(new ResumableUploadProgress(UploadStatus.Completed, BytesServerReceived));
        }
    }
    catch (TaskCanceledException ex)
    {
        Logger.Error(ex, "MediaUpload[{0}] - Task was canceled", UploadUri);
        UpdateProgress(new ResumableUploadProgress(ex, BytesServerReceived));
        throw ex;
    }
    catch (Exception ex)
    {
        Logger.Error(ex, "MediaUpload[{0}] - Exception occurred while uploading media", UploadUri);
        UpdateProgress(new ResumableUploadProgress(ex, BytesServerReceived));
    }

    return Progress;
}

public async Task<IUploadProgress> ResumeAsync(CancellationToken cancellationToken)
{
    try
    {
        using (var callback = new ServerErrorCallback(this))
        {
            while (!await SendNextChunkAsync(ContentStream, cancellationToken).ConfigureAwait(false))
            {
                UpdateProgress(new ResumableUploadProgress(UploadStatus.Uploading, BytesServerReceived));
            }
            UpdateProgress(new ResumableUploadProgress(UploadStatus.Completed, BytesServerReceived));
        }
    }
    catch (TaskCanceledException ex)
    {                       
        UpdateProgress(new ResumableUploadProgress(ex, BytesServerReceived));
        throw ex;
    }
    catch (Exception ex)
    {                       
        UpdateProgress(new ResumableUploadProgress(ex, BytesServerReceived));
    }

    return Progress;
}

These are the Fiddler records showing the upload resuming.

Answer 1

After a fair bit of deliberation, I've decided to modify the API code. My solution maintains backwards compatibility.

I've documented my changes below but I don't recommend using them.

In the UploadAsync() method in the ResumableUpload Class in "Google.Apis.Upload", I replaced this code.

BytesServerReceived = 0;
UpdateProgress(new ResumableUploadProgress(UploadStatus.Starting, 0));
// Check if the stream length is known.
StreamLength = ContentStream.CanSeek ? ContentStream.Length : UnknownSize;
UploadUri = await InitializeUpload(cancellationToken).ConfigureAwait(false);

with this code

UpdateProgress(new ResumableUploadProgress(
    BytesServerReceived == 0 ? UploadStatus.Starting : UploadStatus.Resuming, BytesServerReceived));
StreamLength = ContentStream.CanSeek ? ContentStream.Length : UnknownSize;
if (UploadUri == null) UploadUri = await InitializeUpload(cancellationToken).ConfigureAwait(false);

I also made the UploadUri and BytesServerReceived properties public. This allows an upload to be continued after the ResumableUpload object has been destroyed or after an application restart.

You just recreate the ResumableUpload as per normal, set these two fields and call UploadAsync() to resume an upload. Both fields need to be saved during the original upload.

public Uri UploadUri { get; set; }
public long BytesServerReceived { get; set; }

I also added "Resuming" to the UploadStatus enum in the IUploadProgress class.

public enum UploadStatus
{
    /// <summary>
    /// The upload has not started.
    /// </summary>
    NotStarted,

    /// <summary>
    /// The upload is initializing.
    /// </summary>
    Starting,

    /// <summary>
    /// Data is being uploaded.
    /// </summary>
    Uploading,

    /// <summary>
    /// Upload is being resumed.
    /// </summary>
    Resuming,

    /// <summary>
    /// The upload completed successfully.
    /// </summary>
    Completed,

    /// <summary>
    /// The upload failed.
    /// </summary>
    Failed
};

Nothing has changed for starting an upload.

Provided the ResumableUpload Oject and streams have not been destroyed, call UploadAsync() again to resume an interrupted upload.

If they have been destroyed, create new objects and set the UploadUri and BytesServerReceived properties. These two properties can be saved during the original upload. The video details and content stream can be configured as per normal.

These few changes allow an upload to be resumed even after restarting your application or rebooting. I'm not sure how long before an upload expires but I'll report back when I've done some more testing with my real application.

Just for completeness, this is the test code I've been using, which happily resumes an interrupted upload after restarting the application multiple times during an upload. The only difference between resuming and restarting, is setting the UploadUri and BytesServerReceived properties.

resumableUploadTest = youTubeService.Videos.Insert(video, "snippet,status,contentDetails", fileStream, "video/*");
if (resume)
{
    resumableUploadTest.UploadUri = Settings.Default.UploadUri;
    resumableUploadTest.BytesServerReceived = Settings.Default.BytesServerReceived;                 
}                                               
resumableUploadTest.ChunkSize = ResumableUpload<Video>.MinimumChunkSize;
resumableUploadTest.ProgressChanged += resumableUpload_ProgressChanged;
resumableUploadTest.UploadAsync();

I hope this helps someone. It took me much longer than expected to work it out and I'm still hoping I've missed something simple. I messed around for ages trying to add my own error handlers but the API does all that for you. The API does recover from minor short hiccups but not from an application restart, reboot or prolonged outage.

Cheers. Mick.

Answer 2

This issue has been resolved in version "1.8.0.960-rc" of the Google.Apis.YouTube.v3 Client Library.

They've added a new method called ResumeAsync and it works fine. I wish I'd known they were working on it.

One minor issue I needed to resolve was resuming an upload after restarting the application or rebooting. The current api does not allow for this but two minor changes resolved the issue.

I added a new signature for the ResumeAsync method, which accepts and sets the original UploadUri. The StreamLength property needs to be initialised to avoid an overflow error.

public Task<IUploadProgress> ResumeAsync(Uri uploadUri, CancellationToken cancellationToken)
{
    UploadUri = uploadUri;
    StreamLength = ContentStream.CanSeek ? ContentStream.Length : UnknownSize;
    return ResumeAsync(cancellationToken);
}

I also exposed the getter for UploadUri so it can be saved from the calling application.

public Uri UploadUri { get; private set; }

Answer 3

I've managed to get this to work using reflection and avoided the need to modify the API at all. For completeness, I'll document the process but it isn't recommended. Setting private properties in the resumable upload object is not a great idea.

When your resumeable upload object has been destroyed after an application restart or reboot, you can still resume an upload using version "1.8.0.960-rc" of the Google.Apis.YouTube.v3 Client Library.

private static void SetPrivateProperty<T>(Object obj, string propertyName, object value)
{
    var propertyInfo = typeof(T).GetProperty(propertyName, BindingFlags.NonPublic | BindingFlags.Instance);
    if (propertyInfo == null) return;
    propertyInfo.SetValue(obj, value, null);
}

private static object GetPrivateProperty<T>(Object obj, string propertyName)
{
    if (obj == null) return null;
    var propertyInfo = typeof(T).GetProperty(propertyName, BindingFlags.NonPublic | BindingFlags.Instance);
    return propertyInfo == null ? null : propertyInfo.GetValue(obj, null);
}

You need to save the UploadUri during the ProgressChanged event.

Upload.ResumeUri = GetPrivateProperty<ResumableUpload<Video>>(InsertMediaUpload, "UploadUri") as Uri;

You need to set the UploadUri and StreamLength before calling ResumeAsync.

private const long UnknownSize = -1;
SetPrivateProperty<ResumableUpload<Video>>(InsertMediaUpload, "UploadUri", Upload.ResumeUri);
SetPrivateProperty<ResumableUpload<Video>>(InsertMediaUpload, "StreamLength", fileStream.CanSeek ? fileStream.Length : Constants.UnknownSize);
Task = InsertMediaUpload.ResumeAsync(CancellationTokenSource.Token);