On Fedora Linux you start with bash as your default shell. I prefer using zsh so I needed to change the default shell.
We need to use the chsh command to change that.
chsh --shell $(which zsh)
Instead of the full --shell option you can even use the shorthand -s (I prefer to use the full versions for easier understanding).
Thanks to these blog posts for guiding me: