Let's assume this function template:
template <typename F>
void foo(F&& f) {
f("foo");
}
void to_optimize() {
foo([](std::string_view s) {
std::cout << s << std::endl;
});
}
Does, or can, the compiler optimize the inline function away in cases like this? That is, to replace it effectively with just
void to_optimize() {
std::cout << "foo" << std::endl;
}
?

The C++ standard allows any optimization that has no observable effects. As described, optimizing out the function call will have no observable effect. The C++ program has no means of determining whether an additional function call occurred here.
So, yes, a C++ compiler can optimize out the function call. Whether or not it does depends on the compiler and the compiler configuration parameters.