What are the advantages and disadvantages of these two ways of configuring Virtual Machines via PowerShell's Desired State Configuration?
I've heard that Azure Automation DSC has a steeper initial learning curve, but is more reliable in the long term.
Well, let me summarize it for you again (kinda).
Azure Automation and DSC extension both use DSC to achieve desired state to configure VM's. In fact they both use Azure DSC extension to do their job (so its even more fun!).
The main difference is the way you manage those. When you use plain Azure DSC extensions you are forced to configure every VM manually, or with a script of some sort, you don't have centralized reporting, you cannot view state of your VM's, all the management operations are to be carried for each VM individually.
When using Azure Automation you have a central repository that has all the management data you would expect a system of such sort to have. You can store configuration or compiled mof's in Azure Automations, as well as credential assets (and several other things which are not directly related to VM management). Also you can have runbooks performing various activities that can be related to VM management, say turn ON\OFF, provision VM's and configure them, etc
If you are looking to choose which way to go, I'd say it's a no-brainer, go for Azure Automation, you have basically nothing to lose (yes, you would need to invest some time to figure all the pieces out, but it's not like you would need a lot of it). Even if you are generally interested in simply assigning 1-2 configurations to VM's Azure Automation is the way to go, since it is really easy to configure such a scenario and it would allow you to better manage VM's in future in case you would need scale. Also, just like I said, I don't think Azure DSC extension is really stable on its own, in my tests\experiments I've had much better results with Azure Automation