Tidy dataset with multiple items in a value in R

Question

I have a dataset that has multiple items in a value.

col1    col2    col3    col4
1       AA      123     Val1
2       BB      456     Val2, Val3, Val4
3       CC      789     Val3
4       DD      1112    Val4, Val3
5       EE      1314    Val3

I want to tidy the dataset so that col4 only has one value but no value is discarded. In other words, the second record should be expanded to

2       BB      456     Val2
2       BB      456     Val3
2       BB      456     Val4

I'm fairly new to R and I am unsure how this can be done efficiently.

Could not find a solution in my reference materials, possibly because I don't know how to state the problem in R