XML sing URI Reference only alphanumeric

Question

When I digitally sign an XML file with RSA it works perfectly, as long as the Reference URI is not numbers, and I need to sign the XML portion with a purely numeric ID.

I have 2 weeks of research and I can't find the way around.

This work perfectly

<book id="abc123">
    <title>Programming .NET Security</title>
    <author>Adam Freeman</author>
    <author>Allen Jones</author>
</book>


reference = New Reference
reference.Uri = "#abc123"

This does not work, malformed reference element

<book id="12345">
    <title>Programming .NET Security</title>
    <author>Adam Freeman</author>
    <author>Allen Jones</author>
</book>


reference = New Reference
reference.Uri = "#12345"

When the URI Reference is only numbers it doesn't work for me, if it has letters it works, I've tried it in C# and in VB.Net

Framework 3 to 4

Windows 10 Windows 2012 Server Windows 2008