Next time that happens, try this:
Go to Safari’s History menu and choose the well-named “Reopen Last Closed Window.” It looks like this:
And it works.
Right below “Reopen Last Closed Window” is “Reopen All Windows from Last Session.” That one’s great for when you quit Safari and then regret it (this is especially bad when you have lots of windows open). Start Safari up again, choose “Reopen All Windows from Last Session,” and you’re right back in business.
Try these things out before you need them. Muscle memory and all that.