If you have a Valheim dedicated server, and have done all other steps (such as port forwarding, unless you're using a hosting provider) and you've added the server to your favorites in Steam, it probably doesn't show up and just gives a "Not responding" error in Steam. If this happens you can connect directly with the launch options in Steam which will always work.
Example:
+connect ipaddress:port
+connect 95.95.95.95:2456