My simple understanding is OllyDbg is a user mode debugger, which you could use to debug "normal" apps. WinDbg is a kernel mode debugger, which you could use to debug itself.
Is that right?
WinDbg is a GUI debugger which can be used to debug both kernel-mode and usermode programs. It subsumes the functionality of the command-line debuggers, kd (kernel) and ntsd (user). Windbg can be used for live debugging of local usermode processes and remote debugging of kernel and usermode. It can also debug crashdumps after the fact.
WinDbg is a kernel mode debugger developed by Microsoft which can be used to debug Operating System itself on which it is running. Technically, it means it can debug kernel code which is privileged code running in Ring 0.
OllyDbg is a user mode debugger which is capable of debugging only user mode executables such as Exe.
Note that Windbg is a powerful debugger which encompasses the functionality of Ollydbg as well. However, its a command line debugger which beginners find it difficult to dealt with in beginning. Ollydbg is a GUI debugger much similar to Visual Studio debuggers.